A growing number of government departments across India are facing scrutiny over the procurement of so-called “AI-powered” systems, with experts warning that many of these technologies may be nothing more than rebranded automation sold at inflated prices. The controversy has sparked serious concerns about transparency, accountability, and technical due diligence in public spending.
Vendor Mislabeling and Technical Due Diligence Deficits
At the heart of the issue is the rampant misuse of the term “Artificial Intelligence” by vendors and OEMs, who frequently label their products as “AI-enabled,” “AI-powered SOC,” or “intelligent detection systems” without providing any verifiable technical foundation. Officials and procurement teams—often lacking deep technical expertise—are being misled into approving high-value contracts based on vague claims and marketing jargon.
Cybersecurity and technology experts now argue that a structured 100-point AI audit framework is essential for all government procurements involving AI systems. This checklist includes critical questions around model authenticity, data sources, infrastructure, and performance validation—areas where many vendors reportedly fail to provide clear answers.
Investigations and internal reviews have revealed several red flags. In multiple cases, vendors could not specify the machine learning models being used, nor provide details about training datasets, accuracy metrics, or real-world performance benchmarks. Systems marketed as AI-driven were found to rely heavily on static rules and predefined workflows, with little to no adaptive intelligence.
Infrastructure Limits and Third-Party API Opacity
Concerns extend to infrastructure as well. Many vendors claim advanced AI capabilities but lack the necessary computational backbone, such as GPU-based processing or scalable cloud environments. Questions around whether systems run on genuine AI models or simply call third-party APIs without transparency have further complicated the issue.
Another critical gap lies in data governance and compliance. Experts highlight that several AI solutions do not clearly define how data is collected, processed, or stored—raising potential violations of India’s data protection regulations. In some cases, government data may even be used to train shared models without explicit consent, posing significant national security risks.
The audit checklist also emphasizes the need to examine LLM usage, prompt engineering, token consumption, and agentic AI capabilities—areas that are often completely absent in vendor disclosures. Without clarity on these aspects, it becomes nearly impossible to distinguish genuine AI systems from superficial implementations.
Validation Deficiencies and Rule-Based Redundancy
Equally concerning is the lack of testing and validation mechanisms. Many deployments reportedly bypass rigorous testing environments, lack model version control, and do not offer rollback options in case of failure. This raises serious questions about reliability, especially in critical sectors such as law enforcement, defense, and financial systems.
Financial opacity is another major issue. Vendors often bundle AI costs without clear breakdowns, making it difficult for procurement bodies to assess whether they are paying for actual intelligence or mere branding. Hidden costs related to token usage, cloud compute, and licensing further inflate long-term expenditure.
Mandatory Tender Audits and Live Demonstrations
Experts are now calling for mandatory inclusion of AI audit frameworks in all government tenders, along with independent technical evaluations and live demonstrations. A key recommendation is simple yet powerful: vendors must prove that their system can make decisions that cannot be replicated by rule-based logic.
As India accelerates its digital transformation, the stakes are higher than ever. Without stringent checks, the promise of AI risks turning into a costly illusion—one where public funds are spent not on innovation, but on cleverly packaged deception.
The message is clear: in government procurement, “AI-powered” must be proven—not promised.
Algoritha Security Pvt. Limited offers independent AI Audit services as a trusted third-party partner, helping governments and enterprises verify the authenticity of “AI-powered” solutions. By conducting deep technical due diligence—covering models, data, infrastructure, and performance—Algoritha ensures value for money, prevents vendor misrepresentation, and safeguards procurement processes from inflated claims and fraudulent technologies.
For organizations seeking such services, you may contact triveni@algoritha.in.
