Introduction

The Microsoft Azure has a lot to offer when it comes to Cognitive Services, trying out them is really fun. Here in this article, we are going to see how we can detect and read texts from an image and show it in the display using the Cognitive Service Computer Vision AI. The service Computer Vision has it’s own machine language running which helps us to do our task in an easy and effective manner. It uses the technology Optical Character Recognition and extracts the words or text to a readable format. All we have to do is, using the service and write some custom codes as per our requirement. Sounds good? If yes, let’s just skip the introduction and jump on to the real fun.

Background

I love quotes, and I strongly believe that a single quote can do an impact in your life sometimes. One of my hobbies is collecting the images which contain quotes in it. Now, with the help of Azure Computer Vision cognitive service, I thought of creating an application which can get the quotes from the image and save it. You can easily think of many other usages of this service, below are some of them.

1. Using the translator, which scan the image, read the text and translate
2. Document creation from the images, which will be useful to save hours
3. A Google Glass, which capture what you see and translate the content to your native language
4. A digital medical prescription from the actual one

Imagine that you need to create a solution which can save all of your medical prescription contents in a table in a database, so that you can easily sort/filter and show them in a UI. You can even create a mobile application to do that. The possibilities are countless. Now let’s build one of them.

Prerequisites

Before we start, we should have the following things in our hand.

1. A valid Azure Subscription
2. A good Code editor, I prefer VSCode or Visual Studio

Source Code

The source code be downloaded from here.

Let Us Start

As I mentioned earlier, we will be using an Azure service to build our application. Azure is a cloud computing service created by Microsoft, for the last few years there are significant growth in Azure. The services provided in Azure can be found here.

Create the Computer Vision Resource in Azure

To create the Azure resource, login to the Azure portal and click on the +Create a resource menu. Once the window loads, you can search for the resource Computer Vision as shown in the preceding image.

Now, click on the create button and fill the form.

It may take a few minutes to finish the development of the resource. Once it is finished, we can go to the resource and get our keys from the Keys section under Resource Management. Save those keys somewhere as we need them while we create our applications.

Develop the Angular Client Application

Here, we will be creating an Angular application with the help of Angular CLI and to create the application we are going to use the command ng new Azure-AI-Image-Text-Reader where the last parameter is our application name. You should also be given a chance to select your CSS and routing option. Please be noted that you can add them later as well.

Once the application is ready, we can create our own Angular service.

detection.service.ts

To create the service, let’s use the preceding command.

Here the name ‘detection’ is our service name. Now we can edit the service code as preceding.

As you can see that we are creating an HttpClient call to our baseUrl which is been configured in our config.ts file. To call the Computer Vision AI service, it is mandatory to pass the subscription key, the one we have created in the Azure portal, to the request by using the header Ocp-Apim-Subscription-Key.

config.ts

Please remember to change the baseUrl and apiKey before you run the application.

home.component.ts

As our service is ready, let’s create our component now using the command below.

Once the component is created, let’s go to the file home.component.ts and edit our code.

Once the component is loaded in the screen, the user will be able to type or paste the URL of an image and submit the same. Before submitting the request, we are validating the input using the FormControl, we are using the required validator and pattern validator which uses to find whether the given url is valid or not. Once the request is been made we are calling our detection service with a post request and once we get the result from the Azure service, we are looping through the result and formatting the data accordingly.

home.component.html

As we have our component logic ready in hand, now it is time to edit our component template. Let’s edit our home.component.html file as follows.

We use Angular Material design to develop our Angular application, the material component mat-card is a card component which can act as a container, apart from the mat-card we are using the components mat-form-field, matInput, mat-error, mat-card-image in our home component. You can use the command npm install –save @angular/material @angular/cdk @angular/animations to install the material in your application.

Once the component loads, we are giving an option to submit a valid image URL, if the URL provided is not valid, we are disabling the submit button. Once the user submit the request, then we are calling our detection service. Sounds good?

Output

Below are the screen print of our application.

Now we can test this application by giving the URL of the below image.

Once you submit the URL, you should be able to see an output as below.

Demo

The application demo can be viewed here.

Conclusion

Wow!. Now we have learned, what is Azure Computer Vision AI and how to create Azure Computer Vision Cognitive Service. We also learned to create an Angular application which reads the text from images. I hope you found this article useful. I can’t wait to see what you are going to build with Azure Computer Vision AI. You can also check out the other options we have in Microsoft Azure Computer Vision AI here, I am sure you will love playing with it.

Your turn. What do you think?

Thanks a lot for reading. Did I miss anything that you may think which is needed in this article? Could you find this post as useful? Kindly do not forget to share me your feedback.

Kindest Regards
Sibeesh Venu

Introduction

The Cognitive services are always a great topic to consider writing, especially when there is a mix of Machine Learning. If you are totally new to the Cognitive services, consider reading my posts wouldn’t be a bad idea. Here in this article, we are going to use the Microsoft Custom Vision AI, which is a cognitive AI service. With the help of Custom Vision AI, we can easily do the below things.

1. Upload image
2. Tag the images
3. Train the model
4. Evaluate the result

The Custom Vision service uses the machine learning to classify the images we are uploading. The only thing we are required to do is, just specifying the correct tag for each image, you can also tag thousands of images at a time. The AI algorithm is super powerful to give us better accuracy and once the model is trained, use the same to classify new images according to the needs of your app. Let’s skip the introduction and start doing some amazing things. Let’s just start then.

Background

I have a habit of writing article about the things I learn, though I had worked with Cognitive Services before, recently I had a chance to work with Custom Vision AI service, so thought of building something with that. Here in this article, we are going to create an Azure Custom Vision AI which can,

• Scan for real objects
• Identify whether the object is a car or not
• If that is a car, then check which is the car manufacturer

We will be creating a Custom Vision AI model, which we will train with thousands of images. I hope it helps.

Prerequisites

To do the wonderful things, we always need some prerequisites.

1. Account with Custom Vision AI, you can either use the free subscription or your Azure account
2. A database of images for training the model, the same can be downloaded from here
3. Good internet connection, as you will have to upload thousands of images
4. I have already collected a large dataset, which is more than enough to get started, you can download it from OneDrive here

Creating Your Custom Vision AI Model

I assume that you had already gone through the Prerequisites and followed the instructions. If not, please check again.

To get started here, please go to the https://customvision.ai/ and,

1. Log in with your credentials
2. Creating your project
3. Add as many relevant images possible and tag them appropriately
4. Once you tag them properly, you can train your model, if everything goes well, you should see an output as below.

Custom Vision AI Train Output

Please make sure that you are checking the precision, if it is a good number, then you are good to go. As many images you upload and train them with the correct tag, the precision value get increased.  So feel free to add images and see the output yourself.  Below is the count of the images which I trained to my model.

When you train again with the new data, you can see that a new iteration is getting created which is really handy if you need to go back to the previous state, if the precision value is getting decreased.

You can also do a quick test with the UI provided. Please click on the Quick test button and upload an image, let’s upload an image of a bedroom.

Bedroom Test

Well, that was easy. Now let’s upload an image of a Porsche car.

Porsche Car Result

Have you noticed the values returned? The custom vision API is saying that the image is 99.8% is a car and 99.3% is a Porsche car, which is really impressive. Now, what about BMW?

BMW Result

As our Custom Vision AI is working as expected, now we can go ahead and grab our keys which are needed when you create an application to use this Custom Vision AI. Please click on the setting symbol on the right side of the page, and note down your project id and prediction key. You will also need a prediction URL, which you can get from the Performance tab.

Conclusion

The possibilities of cognitive services and machine learning are limitless, you can think of using this in many of our real-life scenarios. If possible, please share me your thoughts on where are all the places we can use this Custom Vision AI service. In this article, we have learned,

• What is Custom Vision AI Service
• How to work with Custom Vision AI service
• How to train and test our Custom Vision AI model

Is that all? No, in the next article we will create an application which can be used to test it in real time.

Your turn. What do you think?

Thanks a lot for reading. Did I miss anything that you may think which is needed in this article? Could you find this post as useful? Kindly do not forget to share me your feedback. What are you waiting for? create your model and train.

Kindest Regards
Sibeesh Venu