Machine learning & license plate recognition: An ideal partnership

The number of vehicles is increasing significantly. This increase has invoked the necessity of an automatic surveillance system. The License Plate Recognition and detection is a key technique in most applications related to vehicle movement. No doubt, there are various techniques and algorithms to detect and recognize License Plates, but image processing-based solution in form of Machine Learning is the most accurate because of the following reasons:

  • A lot of information is stored in an image that can be utilized from varied perspectives
  • Ability to solve complex visual recognition task
  • Identify vehicle license plates in an efficient manner without the need for major human resources

The Machine Learning framework of Apple has opened the doors of various businesses to come up with more applications that are intelligent enough to perform cumbersome and difficult functions with complete ease. And one such application that can be designed using this formidable framework is the License Plate Recognition System.

With this blog, we will help you in building an application that can accurately lead to License Plate Recognition, which can prove its worth across varied industries such as security, transport, and insurance.

The concept of License Plate Recognition

License Plate Recognition is an effective process of detecting and recognizing the number plates using the concept of Optical Character Recognition (or OCR) on images. It is usually a three-stage process that goes as follows:

1) License Plate Detection: Considered as the most important step of the system, it determines the position of the license plate. Vehicle image goes in as an input, which returns the license plate as the output.

2) Character Segmentation: At this stage, alphanumeric characters from the license plate are extracted and segmented into individual images.

3) Character Recognition: Each individual character that was segmented earlier, gets recognized at this stage. And Machine Learning is used to perform this step.

Machine Learning is a subset of artificial intelligence (AI) that paves the for the system to automatically learn and improve from the encounters, thereby delivering relevant yet better experiences. Core ML is a framework put forward by Apple a few months ago that supports features like face tracking, face detection, landmarks, text detection, rectangle detection, bar code detection, object tracking, and image recognition.

The basics of core ML model

Core ML supports a number of trained machine learning models that get integrated into the app. And this includes tree ensembles, neural networks,  support vector machines, and generalized linear models.

Core ML acts as a core that is tightly integrated and supports the Vision (image analysis), Natural Language Processing, and GameplayKit frameworks.

Building a license plate recognition system

No doubt, License Plate Recognition systems can be built without using machine learning too, with techniques like template matching, feature extraction etc. However, using machine learning to develop an LPR system improves the accuracy of the system because of an improved training process.

  • The process initiates with choosing a framework in iOS that uses neural networks for image recognition – the ideal ones are Tesseract and SwiftOCR.
  • If you seek for better accuracy, then go ahead with SwiftOCR. And if you are after the speed, Tesseract is a preferred option.
  • However, if you are looking for ideal accuracy with less consumption of memory, go ahead with Apple’s latest launched Vision Framework that contains features like face recognition, text, barcodes, QR codes, etc.

Vision framework for license plate recognition

Vision is a framework that uses high-performance image analysis and computer vision techniques to identify faces, face landmarks, text, rectangles, barcodes, and objects:

The unique features of the Vision Framework

  • Rectangle Detection
  • Face Detection
  • Barcode Detection
  • Text Detection
  • Object Tracking
  • Image Alignment
  • Image processing with Core ML Model

The process of building LPR with Vision Framework

Two of the following functions are to be performed using a vision framework:

1) Detect regions of visible text and split the word image into characters
2) Process the images with Core ML model

Cut the images with vision “VNDetectTextRectanglesRequest”. However, this leads to the detection of just characters and not its recognition. For recognition, a Core ML trained model is required as following:

Core ML model creation

From the above-mentioned step, it was possible to recognize just the text of the license plate. However, the entire license plate needs to get recognized for smooth processing. So, to make that happen, train the model through Keras – an open source neural network library written in Python for fast experimentation with deep neural networks. Build the model in python and add the following frameworks:

  • Keras to create the model from a dataset
  • Image to fetch the pixels of the character image
  • Numpy for a high-performance multidimensional array object
  • Core ML tools to convert into ML model
  • Sklearn for splitting the dataset for training and testing

The process

In this step, there is a requirement of data in form of images of characters

28*28, which will be trained.

  • Gather data from the web and also collect around 50 images of number plates
  • Slice the characters to convert them into 28*28 resolution from any one number plate image.
  • Turn the process automated by building an app in iOS. Generate thousands of character images with the help of vision framework.

The process explained in a nutshell

  • Train model in your preferred framework.
  • Use coremltools python package to convert the model to .mlmodel
  • Use the model in your built app.

Applications of the License Plate Recognition System

Verification of authenticity of abandoned vehicles

This is a valuable feature for the police forces for law enforcement purposes. This feature can be leveraged by the police forces for Law Enforcement purposes. Here, the application can enable them to quickly check the authenticity of number plates installed on abandoned vehicles via mobile devices.

Automation of electronic toll collection

Regulated entry of vehicles through toll barriers can be automated with LPR effectively. Number plates listed in toll collection database can be recognized with assorted use of LPR and an access control system.


Detecting and recognizing the vehicle’s license plate is necessary because of the rising number of vehicles and it’s going beyond human’s ability to complete this cumbersome task. License plate recognition though Machine Learning is an ideal and effective method to detect and recognize the license plate of vehicles in a speedy and accurate manner.

Author avatar
James Wilson

I am James Wilson and I love writing on various topics, be it a technical like web/ mobile app development or non-technical like sharing my experience on how to manage both home and work at the same time. I have started out as a quality analyst and moved onto business analysis, where I discovered my passion for information architecture, wireframes, and user experience. I have solid experience in business analysis and am now interesting myself in day researching rapid prototyping methods.

Post a comment