Online learning platform Coursera recently reported that Indians constituted the highest number of subscribers for artificial intelligence and machine learning courses. Now hear this: many companies are seeking individuals with expertise in Reinforcement Learning (RL), which is more advanced than Machine Learning (ML). RL models are used for simulations in autonomous vehicles and assisted driving, for instance. The other application areas are those that use computer vision, natural language processing (NLP), and robotics. Unlike machine learning, RL models can be built without the need to have a lot of data. The RL algorithms are capable of learning on their own – in less time than it would take to train a machine learning model. What’s more, RL models can be built using a variety of (open source) frameworks, thereby offering flexibility.
DIGITAL CREED met Olivier Klein, Head of Emerging Technologies, AWS- APAC at the Amazon AI Conclave in Bengaluru last month, to understand the future path for machine learning, deep learning, and artificial intelligence.
Read our post-show report about the Amazon AI Conclave here.
Olivier Klein, Head of Emerging Technologies, AWS-APAC
“We see a lot of advancements being made in deep learning and there are three major areas: computer vision, Natural Language Processing (NLP), automation & robotics,” said Klein. “Using the tools and services provided by Amazon, it has become far easier to create machine learning models for these applications. That was not the case ten years ago.”
Klein showed a demo of a computer vision model during his keynote at the Amazon AI Conclave. It was built using an open source framework (although the same could also been built using an Amazon framework).
“I didn’t put much effort into getting these demos running. It shows that computer vision has become something that everyone can use. With Amazon Rekognition I don’t even need to build a model,” said Klein.
He said that huge advancements are being made in Natural Language Processing and Speech Recognition – both of which use deep learning.
“We try to democratise that in many different ways like fully managed services and through Amazon SageMaker — which has a lot of built-in algorithms to build your own NLP models and your own language capabilities,” said Klein.
Reinforcement Learning is better
There have been a couple of challenges with machine learning and RL addresses many of those challenges. RL is being used in the areas of Automation & Robotics, especially for applications like assisted driving and autonomous vehicles. It is also used for developing models for warehouse automation and medical image analysis.
According to documentation on the Amazon website, RL is particularly suitable for complex, unpredictable, environments that can be simulated, and where building a prior dataset would either be infeasible or prohibitively expensive: autonomous vehicles, games, portfolio management, inventory management, robotics or industrial control systems. RL-based control is more cost effective than traditional rule-based systems. For instance, researchers have shown that applying RL-based control to HVAC systems can result in 20% – 40% cost savings compared to typical rule-based systems, not to mention the large reduction in ecological footprint (Source: https://amzn.to/2CMs7xi)
A quick primer on Reinforcement Learning
“We made reinforcement learning capabilities available at our AWS re:Invent 2018 event. We introduced RoboMaker services for building robotics applications. You train the robots using reinforcement learning (RL),” said Klein.
AWS RoboMaker is a Cloud robotics service that makes it easy to develop, test, and deploy intelligent robotics applications at scale.
RL was a research topic a few years ago and only the scientists and techies spoke about it. It is not easy to use, and one needs to understand how a simulation environment works. One also needs to build a technology stack to make it work.
Amazon has now made it simpler for organisations to use RL through Amazon SageMaker RL, which it launched at AWS re:Invent 2018.
Launched at AWS re:Invent 2017, Amazon SageMaker is helping customers quickly build, train and deploy ML models. Amazon SageMaker RL extends the advantages of Amazon SageMaker to reinforcement learning, making it easier for all developers and data scientists regardless of their ML expertise (Source: https://amzn.to/2CMs7xi)
Klien demonstrated an RL algorithm featuring a stick man runner learning to stand upright on its own, and start running. This was done with a few clicks.
“The key thing here is that we did not specify how running works and how one can run or stand on one’s own. None of that was put into the algorithm. The only thing that was put into the algorithm was, if you fall, it is bad. If you don’t fall it is good. From there you start iterating. In just a few hours of simulation, that stick man learned how to run,” said Klein.
From that we understand that the capabilities of RL and different and far more progressive than those of traditional machine learning. As we mentioned earlier, machine learning requires a lot of data to train the models. However, RL can be done with “zero data” informed Klein.
“You just tell the model the way it has to work and the model will learn. And as it simulates it creates data on which it learns itself,” he said.
RL Applications
One area that we see a lot of potential for this is autonomous vehicles. A company called TuSimple is doing research in this area. TuSimple is using RL and ML models to build what it calls “the safest self-driving truck” with a 1,000 meter perception range. And you can read more about this here: https://prn.to/2GV2vCr
Mobileye (an Intel company) is also building machine learning models for assisted driving and autonomous vehicles. The company is using computer vision and reinforcement learning in those models. You can read more about this here: https://www.mobileye.com/
RL enables one to simulate autonomous driving and assisted driving in a simulator — the model can be perfected before an autonomous vehicle is put on the street.
Both Mobileye and 2 Simple are using Amazon machine learning services.
The writer was invited to and hosted at the Amazon AI Conclave in Bengaluru