A recent tax tribunal order, hastily withdrawn after citing non-existent court judgments, has sparked concerns over the possible use of generative AI—a groundbreaking technology reshaping the world but notorious for sometimes fabricating information.

In December, the Bengaluru bench of the Income Tax Appellate Tribunal (ITAT) issued an order referencing three Supreme Court judgments and one Madras High Court ruling—none of which exist. Within a week, the order was swiftly withdrawn, citing “inadvertent errors.”

What is the Case?

The case in question is Buckeye Trust v. PCIT (ITA No. 1051/Bang/2024). Buckeye Trust, a private discretionary trust established in 2018, was settled with investments worth ₹669.27 crores, primarily through the transfer of interest in a partnership and unlisted shares. The trust filed its return for AY 2018-19, declaring nil income. The Assessing Officer (AO) accepted this in the assessment order under Section 143(3) of the Income Tax Act.

However, the Principal Commissioner of Income Tax (PCIT) reviewed the assessment order and found it erroneous and prejudicial to revenue. Typically, transfers without consideration exceeding ₹50,000 to non-relatives are taxable in the recipient’s hands. However, Buckeye’s lawyers argued that an interest in a partnership does not qualify as “property” under tax law, among other submissions.

After evaluating the arguments, the tribunal ruled that an interest in a partnership qualifies as a form of “share” and is therefore taxable. In its order, the tribunal cited precedents from the Supreme Court and various High Courts—later found to be non-existent or misattributed.

AI Hallucination: Fabricated Case Laws

Generative AI, while transformative, is prone to “hallucinations”—instances where AI tools like ChatGPT and Gemini generate responses that appear factual but are entirely fabricated. This flaw seemingly played a role in the tribunal’s now-revoked order.

The order cited the following three judgments:

• K. Rukmani Ammal v. K. Balakrishnan (1973) 91 ITR 631 (Madras High Court)
• S. Gurunarayana v. S. Narasinhulu (2004) 7 SCC 472 (Supreme Court of India)
• Sudhir Gopi v. Usha Gopi (2018) 14 SCC 452 (Supreme Court of India)

However, the first two case citations do not exist, while the third citation actually refers to a different case—K. Subba Rao v. State of Telangana.

A fourth citation, 57 ITR 232 (SC), does correspond to an actual case—CIT vs. Raman Chettiar—but it pertains to a Hindu Undivided Family and is completely unrelated to the Buckeye Trust matter.

It appears that the tax department’s representative (DR) may have relied on a generative AI tool like ChatGPT to find judgments that favored the department and incorporated them into his arguments. Meanwhile, the tribunal appears to have copied and pasted the case laws from the DR’s submission without conducting proper due diligence.

ITAT Retraction and Re-examination

Legal experts uncovered the fabricated citations, prompting the ITAT to revoke the order under Section 254(2) of the Income Tax Act and schedule a fresh hearing for February 19, 2025. The tribunal moves to correct the error and uphold fairness, where both parties are expected to present new arguments in the case’s re-examination.

The Buckeye Trust case serves as a stark reminder of the risks associated with unverified AI-generated content in legal proceedings. While AI can be a valuable tool for legal research, its tendency to fabricate citations poses significant challenges if left unchecked.

As the ITAT re-evaluates the case, it highlights a broader lesson for legal professionals: technology should enhance, not replace, meticulous legal scrutiny. Moving forward, courts and practitioners must adopt stricter safeguards and verification protocols to prevent similar errors, ensuring that judicial decisions are based on authenticated precedents rather than AI-generated misinformation. As AI continues to integrate into legal research, regulatory bodies must also establish clear guidelines on its responsible use to prevent such mishaps in the future.