Are Google’s AI models able to recognize accurately maker organizations and businesses?
Earlier this year the Colorado Maker Hub initiated a first phase of a project, Making Connections. The overall project goal is to see how much the Hub will be able to leverage an organization’s website and machine learning models to maintain our directory. Our directory is a key component to our mission of connecting and communicating the Colorado maker community. Its accuracy should be high for our community and user to feel confident in using it. Users should be able to find community members by those nearby, those that make what they are interested in, those that offer equipment to make, a community to join or resources with which to create. This is not an easy task to maintain what each organizations does…and with 1300+ strong already in the community directory we discovered how hard it might be to maintain manually. So the goal was set…and the project was initiated to see what was possible.
We had identified an organization’s primary website as our target source for data and content for which to run machine learning models. Our assumption was that an organization’s primary website should be their best communication of what they do. The content and images that are portrayed on their website should be the primary source to help classify what they do. With a snapshot of our directory earlier in the year, we got started!
We knew we needed to pull content and images from an organization’s primary website as our first step. To share in the learnings we decided to setup a high school internship within the Pikes Peak Business & Education Alliance (PPBEA) for the Spring 2019 semester. We had two high school students from Lewis Palmer School District come onboard as a 60-hour internship. The student’s focus was to create Jupyter Notebooks using Python that would pull contact information, content and images from an organization’s primary website. From the output of these Notebooks their three adult mentors (Kirsten Cox, Santiago Norena and myself) ran the images through Google’s CloudVision to label objects and then ran both the website content and image results, separately, through Google’s Natural Language Processor (NLP). After all this processing across 1341 organizations in our directory snapshot, we were able to classify 72% of the organizations to one or more of Google’s Categories. But now the analysis begins…how accurate were Google’s AI models? Are they informative enough to connect and communicate the community members? Will we need to create our own models to better identify maker community members for their Types of Making and Community Roles? Give us your feedback.
For an example of how we currently identify maker community members manually by their Types of Making and Community Roles you can browse our current directory. Below are examples of the high-level and low-level categories that Google’s models are recognizing maker organizations as:
Click here for a list of all the counts of low-level categories in our first phase of our project.