You want a smart Edge that’s a gatekeeper

V Padmanabhan, Vice President, Engineering and South Head, GlobalLogic, talks about the journey of data and how Big Data is delivering the goods off late and that’s also powering the whole concept of Machine Learning.

We have multiple core themes of work. One is around cloud native or hybrid native. We deal with whole tech products or Greenfield products but born in the cloud. Another side of the offering is around Big Data. The entire data stack. That’s wherever you need to have a strong embedded ecosystem in the front end, a strong Edge or a strong IoT (Internet of Things). We take a turnkey type of approach. Then a big part of the pie is user experience. We have multiple approaches. One is a typical agency or studio approach. We do user, empathy and persona studies and sometimes recommend business model changes, like say mobile may not be suitable in a particular situation. We are also into business consulting. A majority of our work is very closely connected to customers’ business outcome. Most of the platforms we develop directly impacts revenue, brand and market reach.

The journey from data to Big Data

From a historic data availability standpoint and our ability to use it, what you're seeing is banking and healthcare having good data collection purely because of compliance needs that have been there from the beginning. The consumer industry started collecting data properly at the start of e-commerce. Auto industry started getting into the space in the last five six years with the on board dongle. Apart from that it was largely retail data, shops and stuff. Now you want to solve modern world problems with historic data. That is where the application comes into play. One is quality of data and the second is applicability.

Over the last three four years, Big Data delivering results have increased, especially in marketing, for example micro targeted lead generation campaigns. It has also increased in healthcare, travel and leisure. Telecom was always using it for rating and pricing. So, accuracy has increased. Most importantly, culturally people have started going towards data driven decisions. Earlier people used to look at data with suspicion, but now people are asking for it before they make a decision. All these things will push towards better data collection. But interestingly, the poorest quality of data is in CRM. Because CRM comes with 30 fields, nobody has the patience for anything else: Customer name, first name, last name, contact, location etc. But nowadays we look at, especially in B2C, things like demography, age, family patterns etc.

Data-ML feed off each other

There is a lot of investment going on in the data engineering side to get ready for the data deluge that is on the way. That is also helping Machine Learning because the more the data there is the better it becomes, they feed into each other. There is also a democratisation of ML that is going on and that is making things simpler. When I start coding, the open source resources I used in a project were maybe 5%. At that time people were not contributing. Then it became 15-20% and Apache picked up. Today open source libraries are quite rich. Productivity has increased and costs have come down. The ML community is quite strong and has gone mainstream in the last 2-3 years. You don’t have to re-invent models any more. There are so many publicly available ones. You can choose whatever learning you want, unsupervised or supervised, regression studies, binomial classification...

Building the right talent pipe

We have to build talent and also do retraining. What we see is when you are very fresh on mathematics and when you teach coding, you are good on the data side. We have GLADE (GlobalLogic Academy for Data Engineering) and various other academies to deal with each problem. We found a way to collaborate with academia, especially in the Big Data area. Customers come back to us and say, don't take a lot of time with the data. Take 30-45 days, but tell us what you can do this data. College kids do great on the R&D side and they're very close to mathematical problems. In this regard colleges set up labs and are good at providing recommendations. It’s a very transparent process.

Sometimes these kids are bright enough for us to absorb into the ecosystem. Campus to corporate is becoming a big differentiator for us in terms of attracting talent. We also partner with Coursera and Udemy. It is very important that we are getting fresh talent to solve our problems. We have set our talent pipeline accordingly. Data engineering is open to only software engineers, but data science has phenomenally opened up to all skills. For example, in ML itself, there is no code there; it is just the machine picking up patterns.

A look to the future

GPUs have a long way to go before they reach a limit. For example, some of the smartphones are actually they are multiplying the GPU capability by say 12x. Unlocking a phone which used to take 2 seconds, they want to bring that down to nanoseconds, that’s using the GPU. Then you want to look at the quantity of data too. You want a smart Edge which churns streams of data, keeps looking like a gatekeeper. It can look for patterns and maybe declare a Red Alert when things go down and shut down a factory which it is powering. Edge compute is becoming faster, but the speed is used to analyse larger and larger streams of data. Right now it’s about higher velocity but lower volumes of data. Edge needs velocity processing. For volume processing you go to the cloud.