Congratulations on finishing module 1! Well done!!!
So pleased it was helpful.
Great question. This will be answered as you progress through the modules and module 2 is particularly useful for understanding an overview of the process. The key things for multiple solutions being integrated are the data itself, and the tech stack for how different solutions communicate.
1. Data – So, if you are using multiple data sources, they have to be sorted, structured, cleaned and processed to be ready for AI algorithms, and in most cases you are not looking at a static data dump but a living breathing data source that is constantly updated (e.g. social media data or even patient record data). Then they need to be streamed in real time into the tech stack, or updated with batch data uploads.
2.Tech stack. To do this you need a tech stack that ingests the data from the different data sources, and sorts, cleans, processes etc etc automatically (but the data scientists working on the project write the algorithms to do that, and the big data engineers and programmers make it seemlessly work together).
I don’t want to scare you with a really detailed example at this stage so I have uploaded simplified example of a tech stack we created many years ago for omni-channel analytics for a client. This is obviously a simplified version and doesn’t have all the depth but it will give you an overview of how a lot of the parts fit together.
Tech Stack Example
This is why AI projects are costly to create. They are very complex and require a lot of moving parts and fairly large expert teams. This project had a team of 27 working on it (strategists, data scientists, big data architects, full stack developers, computational linguists etc) and took 18 months to finish and all were working solely on that project full time. Although having said that, I did recently ask one of our big data engineers if that project would be faster today due to advances in tech and he said absolutely it would be significantly faster due to many of the new tech tools we can use in the stack. I am not saying it would only take weeks now (as that is not the case), but it would not take as long as it did 10 years ago.
So, all the solutions you see in the game plans will have sophisticated engines like this behind them in their tech stacks and will have cost millions to hundreds of millions in VC money to create. That is why to create an AI project – one with a tech stack at least – you are talking fairly serious money. That is why I wanted to make existing solutions available for teams with smaller budgets who cannot afford hundreds of thousands to many millions to do this – enter the game plans.
The question you have about is it purely supervised learning or a mixture is very dependent on the project itself, what data is being used and what you are trying to do. So the data scientists will look at that when determining what techniques and approaches are the right ones for the project. So you will have the decision of what types of AI – are we doing machine learning or NLP or a combination etc etc , and then what type of machine learning and then what sub-type and then what technique in terms of reinforcement learning etc. So for something with a lot of images you are probably using deep learning (which is a sub type of artificial neural networks, which is a sub type of machine learning). For market research data (that is not language data analysis) you are probably looking at a decision tree model like random forest. The data scientist is the expert so they will look at what you are trying to do, and what data you have and make that determination.
.
0
Points
