The Ghostwriter in the Procurement Department: Auditing the Machines Now Choosing Your Vendors
Photo: Internet Archive Book Images, No restrictions, via Wikimedia Commons
Somewhere in your organization, an RFP is being drafted right now by a language model. Nobody made that decision in a steering committee. A business analyst with a tight deadline pasted last year's requirements document into a chatbot, asked it to "modernize and expand," and moved on. The output was fluent, comprehensive, and largely plausible. It was also, in places, wrong.
This is the new reality of enterprise procurement in the United States: artificial intelligence is no longer a topic discussed in vendor pitches — it is an active participant in the buying process itself. And most CIOs have no visibility into how, or how extensively, it is being used.
The Invisible Drafting Floor
Talk to procurement leaders at mid-market and enterprise companies and a pattern emerges. Requirements documents are being generated, expanded, and "professionalized" by AI tools. Vendor responses are being summarized and compared by the same tools. Scoring matrices — once a spreadsheet of human judgment — are increasingly populated with machine-generated assessments of proposal language.
None of this is inherently bad. A five-person IT team evaluating forty vendors cannot read forty 80-page proposals with fresh eyes. AI summarization is often the only way the comparison happens at all. But the process has outpaced the governance, and that gap carries three distinct risks.
Risk One: Requirements Invented by a Model
When a language model expands a requirements document, it does not distinguish between what your organization needs and what similar organizations typically ask for. It blends your genuine requirements with patterns absorbed from training data — which means it can quietly insert capabilities nobody requested, omit constraints that are mission-critical, or phrase a preference as a mandate.
We have reviewed AI-drafted RFPs that included data residency requirements contradicting the buyer's actual compliance posture, and others that specified integration protocols the incumbent systems could not support. The language was confident. The documents looked finished. That is precisely the problem: a flawed requirement written by a junior analyst looks like a draft; a flawed requirement written by a model looks like a decision.
The fix is unglamorous. Every AI-assisted requirement should be traceable to a named human owner. If no one on the business side can explain why a line item exists, it does not go to vendors.
Risk Two: Bias With a Compliance Footprint
Scoring is where AI-assisted procurement moves from operational convenience to legal exposure. US public-sector buyers already operate under strict procurement fairness rules, and private enterprises with government contracts inherit many of the same expectations. If an AI tool systematically favors certain writing styles, certain vendor archetypes, or certain phrasings — and large vendors with dedicated proposal teams write differently than small ones — the selection process may be challenged on grounds of arbitrary or discriminatory evaluation.
There is also a subtler distortion. Models trained on existing enterprise content tend to reward the familiar. Proposals that echo standard industry language score well; genuinely novel approaches that break convention can be penalized for failing to match expected patterns. Buyers risk selecting the vendor that writes like everyone else, not the one that solves the problem differently.
Organizations should require disclosure of where AI was used in evaluation, run periodic spot-checks where humans re-score a sample of proposals blind, and document the rationale for every scoring decision — not because the process is presumed flawed, but because the burden of proof now falls on the buyer.
Risk Three: The Vendor Side of the Mirror
The same dynamic runs in reverse. A meaningful share of RFP responses now begin as AI-generated drafts, tuned to mirror the language of the requirements document. Buyers using AI to score proposals written by AI are, in effect, running a machine conversation and calling it diligence. The signal that matters — whether this vendor actually understands your operations — can be laundered into fluent boilerplate on both ends.
The practical countermeasure is to reintroduce friction deliberately: require named references with specifics, demand architecture diagrams and implementation narratives that a model cannot fabricate convincingly, and weight proof-of-concept performance over written responses. The more polished the prose, the less it should be trusted as evidence.
Guardrails Worth Writing Down
A workable AI procurement policy does not need to be long. It needs four commitments:
- Human ownership of requirements. Every specification has a named owner who has read and validated it.
- Disclosure in evaluation. Any AI-assisted scoring is documented, and a human re-review covers a meaningful sample.
- Bias checks on cadence. Quarterly reviews compare AI-assisted outcomes against human-only baselines.
- Legal review of the toolchain. Understand what data your AI tools retain, and whether vendor-confidential information is flowing into systems with unclear terms.
The Strategic View
AI in procurement is not a trend to resist; the efficiency gains are real, and organizations that ban it will simply drive it underground. The competitive question is who builds trustworthy process first. Buyers that pair AI speed with verifiable human judgment will select better vendors, defend their decisions under scrutiny, and spend less time untangling implementations that began with a hallucinated requirement.
The RFP, in other words, did not write itself. Someone — or something — wrote it. Enterprises that know which will be the ones still standing when the contract goes sideways.
