The products reviewed in this report have been built with great care in the selection of data used to train the AI models. The vendors can vouch that no license has been violated using the training data. They have also given much attention to ensuring high quality outputs and a reduction in the presence of “hallucinations,” the euphemism used in the industry for errors. Of much interest to enterprises is the capability to augment a vendor product with private data (from code to documents) to enhance the relevancy of the output to the unique needs of the organization. Ultimately, Omdia still recommends a human-in-the-loop: the presence of humans to check generated output.
Key messages
- AI-assisted software development is a rapidly evolving market, with both startups and established technology vendors involved.
- The impact of OpenAI’s technology has boosted the market for AI-assisted software development. The market players today are mostly agnostic to the foundation LLM they use, picking the best models for the given tasks.
- The range of tasks that AI-based code assistants can take on covers the whole software development lifecycle (SDLC).
- Omdia’s review of academic research on popular LLMs such as GPT-4 and ChatGPT shows that these can offer useful assistance to developers, but generated outputs need to be carefully evaluated.
- Omdia believes that the use of AI-based code assistants has reached a level of proficiency such that enterprises not using this technology will be at a disadvantage.