Like most of us in the procurement tech world, I’ve been thinking quite a bit about generative AI (GenAI) lately. While some critical voices have started to make themselves heard, the hype is still strong and there are few (if any) solution provider briefings that don’t at least mention it.
One of the problems I’ve mentioned before is that the current use cases are far from delivering the transformational value and benefits that everyone is talking about. So, I decided to ask ChatGPT about why I should use generative AI for procurement. The answer was an underwhelming 10-point list of vague talking points like “enhanced decision-making” and “improved vendor management.” The language actually reminded me how many solution provider websites sound like. I guess this reflects what ChatGPT is trained on or that ChatGPT is used to create many of these websites … (ok, it wasn’t the most refined prompt and when I asked more specific questions it did give some more interesting answers).
Anyhow, that prompt didn’t give me anything interesting, so I decided to switch it up and ask why I shouldn’t use generative AI in procurement. This gave me some more interesting answers that are worth discussing.
The list ChatGPT came up with provides the following concerns (the full response is at the end of this article):
- Data Quality and Availability
- Integration Challenges
- High Implementation Costs
- Skill Gaps
- Resistance to Change
- Ethical Concerns
- Dependence on Technology
- Security Risks
- Limited Flexibility
- Short-Term Focus
Some of these assume that you are building your own LLM, which is not really the case for most procurement organizations. Using GenAI as part of a procurement solution will allow you to share the high implementation costs with other customers and relying on the skills of the solution provider can help with the skill gaps — although, at least rudimentary AI, math, and data science skills are becoming increasingly important even for procurement organizations. Integration challenges would also be the solution providers’ concern in this scenario.
The data quality and availability concern is not really unique to GenAI and needs to be addressed for spend analytics and automation purposes regardless of whether you are relying on GenAI or not. Resistance to change is also an ever-present challenge, and the ethical concerns and security risks are pretty obvious and well known. That leaves three that I want to discuss and an obvious one that wasn’t mentioned for some reason.
Hallucinations
Interestingly enough, ChatGPT does not list the problem of inaccuracy or hallucinations. Yes, there are ways of limiting this using good prompts, RAGs (i.e., grounding the response in facts), using high quality data sets for training, and so on. But since LLMs (which is probably the most common type of GenAI used in procurement applications) predict likely and coherent strings of text and do not look up data (not even a RAG will cover every eventuality) the risk of hallucinations remains.
So even if humans are not a guarantee for finding all errors, GenAI should be used with human oversight and with strong governance frameworks (and not where the cost of errors is high). This worry is also related to the first concern that ChatGPT brought up, dependence on technology.
Dependence on Technology
ChatGPT says “Over-reliance on generative AI can lead to complacency in human judgment and decision-making. Procurement professionals may become too dependent on AI recommendations, which could result in overlooking critical factors that require human insight.”
Someone said, and I’m paraphrasing since I can’t recall who said it, that science fiction movies have shown us AI as very accurate but also sounds very robotic. Instead, we have designed AI that sounds very human but isn’t very accurate. We can’t blindly rely on technology, and we need to understand when GenAI (and other forms of AI for that matter) is useful and where humans still are needed.
Limited Flexibility
ChatGPT says “AI models may struggle to adapt to rapidly changing market conditions or unique organizational needs. If generative AI cannot accommodate specific scenarios or fluctuating demand, it may not provide the desired agility.”
This is a real issue in areas like risk management where the need for adaptability and agility is of utmost importance. This is again a case of making sure you understand the limitations of AI and how it’s used by your solution provider.
Short-term Focus
ChatGPT says “Generative AI often focuses on optimizing current processes rather than fostering long-term strategic vision and innovation. Organizations may miss out on broader, transformative opportunities if they become too focused on AI-driven efficiencies.”
Another good point. Perhaps again not limited to use of GenAI but any procurement technology really. We have a saying in Swedish that goes something like “You shouldn’t pave the cow paths” (i.e., just because you’ve always done things a certain way doesn’t mean it’s the right way of doing it, especially considering new technologies that bring new opportunities.
To GenAI or Not to GenAI?
So, even if the original question was “why should I not use generative AI for procurement?” I would say that these are reasons not to use GenAI. I would rather consider these points as considerations and food for thought if you are trying out GenAI in procurement. And, as always, Ardent Partners is here to help you with any questions.
The Full Response From ChatGPT
“While generative AI offers numerous advantages for procurement processes, there are also valid reasons why organizations might hesitate to adopt this technology. Here are some considerations against using generative AI in procurement:
- Data Quality and Availability
Generative AI relies heavily on high-quality, relevant data for accurate predictions and insights. If an organization does not have access to clean, reliable data or lacks proper data management practices, the effectiveness of AI models may be compromised, leading to poor decision-making.
- Integration Challenges
Integrating generative AI tools with existing procurement systems and processes can be complex and time-consuming. Organizations may face difficulties in aligning AI technologies with their current workflows and platforms, which can disrupt operations and delay implementation.
- High Implementation Costs
The initial investment in generative AI technology, including software, infrastructure, and training, can be significant. Smaller organizations or those with limited budgets may find it challenging to justify these upfront costs in relation to potential long-term benefits.
- Skill Gaps
Implementing generative AI requires specialized skills and knowledge that may not be readily available within the current workforce. Organizations may need to invest in training or hire new talent, which can be resource-intensive and time-consuming.
- Resistance to Change
Employees accustomed to traditional procurement processes may resist adopting new technologies, leading to pushback and slow adoption rates. Change management strategies are essential but can be difficult to implement effectively.
- Ethical Concerns
The use of AI raises ethical considerations, such as potential biases in algorithms and decision-making processes. If not properly addressed, these biases can result in unfair supplier evaluations or discriminatory practices.
- Dependence on Technology
Over-reliance on generative AI can lead to complacency in human judgment and decision-making. Procurement professionals may become too dependent on AI recommendations, which could result in overlooking critical factors that require human insight.
- Security Risks
The implementation of AI systems can introduce new cybersecurity risks, including data breaches and unauthorized access to sensitive supplier information. Ensuring data security and compliance with regulations is paramount but can be challenging.
- Limited Flexibility
AI models may struggle to adapt to rapidly changing market conditions or unique organizational needs. If generative AI cannot accommodate specific scenarios or fluctuating demand, it may not provide the desired agility.
- Short-Term Focus
Generative AI often focuses on optimizing current processes rather than fostering long-term strategic vision and innovation. Organizations may miss out on broader, transformative opportunities if they become too focused on AI-driven efficiencies.
Conclusion
While generative AI has the potential to greatly enhance procurement processes, organizations must carefully consider the challenges and drawbacks associated with its adoption. Data quality, integration complexities, cost, and ethical implications are critical factors that require thorough assessment. Balancing the benefits of AI with the need for human insight and strategic vision is essential to harnessing its full potential in procurement. Therefore, organizations should evaluate their specific context and capabilities before deciding to implement generative AI in their procurement strategies.”
