Author : Aastha Raghuwanshi from SVKM’s NMIMS School of Law, Navi Mumbai Co-Author : Aman Singla from Symbiosis International University, Pune
Introduction
The transformation arrives at a moment when justice systems across the world are underimmense pressure. Expanding caseloads delays in adjudication rising litigation costs, andincreasing volumes of digital information have created strong incentives for automation. Legal institutions have therefore turned toward AI-driven systems to improve speed, efficiency, and accessibility.3 The debate surrounding artificial intelligence in lawis thereforenot simply about innovation. It concerns the preservation of constitutional values withinincreasingly automated institutions. Judicial legitimacy is derived not only fromoutcomes but from transparent reasoning, accountability, and independent application of mind. Thesecharacteristics become difficult to preserve when opaque algorithmic systems influencelegal outcomes without meaningful explanation or responsibility.4 This article argues that artificial intelligence should remain institutionally subordinate to human legal reasoning. WhileAImay strengthen administrative efficiency, research capacity, and access to justice, itsparticipation in core adjudicatory functions must remain constitutionally restricted. Thefuture of legal systems lies not in replacing human judgment, but in constructing governanceframeworks that preserve it.
EVOLUTION OF ARTIFICIAL INTELLIGENCE IN THE LEGAL PROFESSION
The legal profession has traditionally been conservative and resistant to technological transformation. Unlike industries rapidly transformed through digitisation, legal practicehas remained heavily dependent upon manual processes, paperwork, and labour-intensiveresearch. Nevertheless, growing competition and increasing demands for efficiencyhavecompelled law firms and courts to gradually adopt AI-driven tools.5
Justice D.Y. Chandrachud has observed that technology becomes relevant only insofar asit promotes efficiency, transparency, and objectivity in governance. According to him, AIshould function merely as a facilitative tool assisting judges in evaluating legal processes and
judgments rather than replacing judicial discretion.6 Similarly, former Chief Justice of IndiaS.A. Bobde clarified that AI may assist with repetitive and mechanical aspects of judicial work but cannot substitute judicial reasoning or human discretion.7
The Indian legal market has increasingly begun experimenting with AI technologies. Oneprominent example is the law firm Cyril Amarchand Mangaldas, which became thefirst Indian law firm to license “Kira,” a machine-learning software developed by Kira SystemsinCanada. Kira is capable of analysing legal documents, identifying risky clauses, conductingdue diligence, and extracting contractual provisions from large databases. Suchtoolssignificantly reduce the time traditionally required for transactional legal work.
The use of AI in law may broadly be classified into six principal areas. 1. Due Diligence
AI assists lawyers in reviewing agreements, contracts, and legal records with greater speedand accuracy. It enables efficient management of electronic evidence, media files, andlegal databases. Machine learning systems can identify problematic clauses, inconsistencies, andcompliance risks within seconds.
2. Prognostication Technology
Predictive AI tools analyse historical case data to estimate possible legal outcomes. Theyassist lawyers in assessing litigation risks, predicting chances of success, and evaluatingcontractual liabilities.
5 Id.
6 Artificial Intelligence in Trial Courts in India: Analysis, SCC ONLINE BLOG (Apr. 16, 2026) 7 Chief Justice S.A. Bobde, remarks at the Judicial Officers’ Conference, Bengaluru, reported in We Have Possibility of Developing Artificial Intelligence for Court System: CJI Bobde, ECONOMIC TIMES (Jan. 11, 2020).
3. Legal Research Mechanisms
AI systems can retrieve and analyse judgments, track judicial patterns, and assist lawyersinidentifying relevant precedents. Such technologies are increasingly integrated intoonlinelegal databases.
4. Documenting Mechanisms
AI enables automated drafting of legal documents, contracts, notices, and pleadings. It alsoassists in organising and managing massive volumes of legal information.
5. Intellectual Property Analysis
AI algorithms can analyse patents, trademarks, copyrights, and other intellectual propertyrecords. They simplify the process of identifying infringement risks and extractingcritical information from complex documents.
6. Electronic Billing
AI-driven software automates billing systems, generates receipts, and manages financial records more efficiently than traditional manual systems.
These developments reveal that AI has already become deeply embedded withinlegal practice. However, the increasing dependence upon AI also raises concerns regardingreliability, transparency, accountability, and constitutional legitimacy
AI IN THE INDIAN JUDICIAL SYSTEM
India has actively explored the integration of AI into judicial administration. The E-CourtsProject Phase III includes provisions for AI-based technological advancement across HighCourts until 2027. Proposed applications include intelligent case scheduling, predictiveanalytics, natural language processing, legal translation, and enhanced data management systems.8
Several AI-based judicial tools have already emerged:
SUPACE
The Supreme Court Portal for Assistance in Court’s Efficiency (SUPACE) assists judgesbyretrieving facts from case files and identifying relevant precedents. It currently has limiteduse in criminal matters before the Delhi and Bombay High Courts.9
Adalat AI
Adalat AI assists in transcribing oral arguments, witness depositions, and courtroomproceedings.10
LegRAA
The Legal Research Analysis Assistant (LegRAA) was developed to aid judges inlegal research and document analysis, although its practical adoption remains limited.11
These systems illustrate that AI can substantially reduce administrative burdens withincourts. Nevertheless, concerns arise when such tools begin influencing substantive adjudicatoryfunctions.12
The Gujarat High Court’s AI Policy expressly prohibits the use of AI for judicial reasoning, order drafting, bail sentencing considerations, or evidence evaluation.13 Similarly, the Punjaband Haryana High Court directed judicial officers not to use AI tools for judgment writingorlegal research.14
These restrictions reflect judicial awareness that adjudication involves far more thandataprocessing. It requires contextual understanding, constitutional interpretation, empathy, andmoral judgment.
THE PROBLEM OF AI-GENERATED “FAKE JUSTICE”
One of the gravest concerns associated with AI in law is the phenomenon of “hallucination,”where AI systems generate fictitious legal authorities that appear genuine.15
The Supreme Court recently confronted a matter involving reliance upon AI-generatednon-existent precedents. The Court emphasised that judicial decisions based on fabricatedauthorities undermine the integrity of the judicial process and may amount to misconduct.16
Similarly, the Delhi High Court dismissed a petition after discovering that it citedfakejudgments and fabricated legal propositions generated through AI systems.17 In anothermatter involving the Akasa Air dispute, the Court observed that portions of the impugnedjudgment appeared to have been drafted using AI-generated reasoning.18
These incidents reveal a fundamental limitation of generative AI systems. Large languagemodels do not “understand” law in the human sense. Instead, they generate responses byidentifying statistical patterns within vast datasets. Consequently, AI systems may producetext that appears authoritative despite being factually false.19
Such hallucinations are particularly dangerous within the legal systembecause courts relyupon precedent, accuracy, and institutional trust. A judgment based on fabricated authoritiesnot only harms litigants but also burdens appellate courts and undermines public confidencein the judiciary.20
THE CONSTITUTIONAL CRISIS OF AI-BASED ADJUDICATION
The core constitutional concern surrounding AI is whether judicial functions maylegitimately be delegated to algorithmic systems. Judicial reasoning requires the applicationof mind. Courts do not merely match patterns between cases; they evaluate facts, interpret law, assess credibility, weigh competing interests, and exercise discretion guidedbyconstitutional morality.21
AI systems, however, function through pattern recognition. They generate outputs byanalysing previous data and identifying statistical similarities. This creates several dangers.22
1. Erosion of Judicial Independence
When judges increasingly rely upon AI-generated reasoning, the judicial role shifts fromindependent reasoning to mere confirmation of machine outputs. Instead of constructinglegal reasoning, judges become auditors of algorithmic conclusions.23
This undermines the constitutional requirement that judicial decisions emerge fromtheindependent application of human mind.24
2. The “Black Box” Problem
Many AI systems lack explainability. Even developers may not fully understand howcertainconclusions are generated. This creates what scholars call the “black box” phenomenon.25
In constitutional democracies, judicial decisions must be reasoned and transparent. Citizenshave a right to understand why liberty was restricted, bail denied, or rights limited. Opaquealgorithmic reasoning violates this principle.26 The House of Lords Select Committee statedthat no AI system substantially affecting human lives should be deployed unless it canprovide satisfactory explanations for its decisions.27
3. Mechanical Justice
Adjudication is deeply fact-specific. AI systems tend to generalise based on patterns foundinprevious cases. However, superficially similar cases may differ significantly in underlyingcircumstances.28
28 Artificial Intelligence in Trial Courts in India: Analysis, supra note 15.
For instance, two bail applications may involve identical offences, yet one accused maybeafirst-time offender caring for an ailing parent. A human judge can evaluate proportionality, dignity, and liberty under Article 21. AI systems lack such contextual humanity. Justicecannot be reduced to statistical prediction.29
4. Loss of Judicial Individuality
Frequent reliance upon AI risks standardising judicial writing and reasoning. Judicial individuality—the unique analytical style of judges such as Justice Krishna Iyer—maygradually disappear.30 Judgments may become formulaic and mechanically uniform, diminishing the richness of legal discourse.
HUMAN RIGHTS AND THE RISE OF ALGORITHMIC GOVERNANCE
The rise of AI has profound implications for human rights. International humanrightsframeworks emphasise dignity, equality, privacy, and non-discrimination. AI systemsthreaten these values when deployed without safeguards.
The United Nations Charter and the Universal Declaration of Human Rights recognisedignity as inherent to all human beings. Similarly, the International Covenant on Civil andPolitical Rights protects privacy and freedom from arbitrary interference.
AI systems create several risks to these rights.
Algorithmic Discrimination
AI systems learn from historical data. If such data reflects existing social biases, the systemreproduces and amplifies discrimination.
This concern is especially serious in societies like India where caste, religion, gender, andsocio-economic inequalities remain deeply embedded.
29INDIA CONST. art. 21; Artificial Intelligence in Trial Courts in India: Analysis, supra note 15. 30 Richard E. Susskind, Expert Systems in Law: A Jurisprudential Approach to Artificial Intelligence and Legal Reasoning, 49 MOD. L. REV. 168 (1986).
The COMPAS system in the United States, used to predict recidivismrisks, was heavilycriticised for racial bias against African Americans. Similarly, predictive policing systemstrained upon discriminatory data risk institutionalising prejudice.
The European Ethical Charter on AI in Judicial Systems warned that AI systems mayviolateprinciples of equality, fair trial, and non-discrimination.
Surveillance and Privacy
AI-powered facial recognition technologies raise major privacy concerns. China’s extensiveuse of surveillance systems demonstrates how AI may facilitate authoritarian control.
Mass surveillance threatens freedoms of expression, association, and assembly. Smart cameras capable of identifying individuals, analysing behaviour, and collecting biometricinformation fundamentally alter the relationship between citizens and the State.
International human rights instruments recognise privacy as a foundational right. Yet AI-driven surveillance increasingly normalises intrusive monitoring.
The Right to a Human Judge
The European Convention on Human Rights implicitly recognises the right to adjudicationbya human judge. Judicial determination involves moral evaluation, empathy, and conscience—qualities absent in algorithms. Automated justice risks reducing individuals into data pointswithin predictive systems
JURISPRUDENTIAL DIMENSIONS OF ARTIFICIAL INTELLIGENCE
The relationship between AI and legal reasoning has long fascinated scholars. Early researchfocused on legal information retrieval systems such as LEXIS, which allowed users tosearchdatabases using keywords.31
Later developments introduced expert systems capable of mimicking expert reasoning. Thesesystems possessed three key characteristics:
1. Transparency
2. Heuristic reasoning
3. Flexibility32
Expert systems rely upon knowledge acquisition, knowledge representation, and knowledgeutilisation.33
Several experimental legal AI projects emerged, including TAXMAN, JUDITH, MITProject, ABF System34 and many.
Despite decades of research, no fully operational expert system capable of replicatingcomprehensive legal reasoning has emerged.
Professor Bryan Niblett famously argued that successful legal expert systems may contributemore to jurisprudence than jurisprudence contributes to them. 35This observation reflectsadeeper reality: AI forces legal philosophy to confront foundational questions regardinglegal reasoning, discretion, interpretation, and legitimacy.
Law is not merely a set of logical rules. It involves moral judgment, interpretative flexibility, social understanding, and evolving constitutional values.36
INTERNATIONAL REGULATORY RESPONSES Different jurisdictions have adopted varying approaches toward regulating AI. European Union
The European Union has emerged as a global leader in AI ethics and regulation. It adoptedthe Ethical Charter on the Use of Artificial Intelligence in Judicial Systems andhasdeveloped extensive AI governance frameworks. The GDPR imposes significant restrictionson automated decision-making and requires meaningful human oversight.37
1. AI should benefit people and the planet.
2. AI must respect human rights and democratic values.
3. AI systems must remain transparent.
4. AI must operate safely and securely.
5. Developers must remain accountable.
China
China has adopted an aggressive AI strategy focused on global technological dominance. Itsapproach prioritises efficiency and surveillance over privacy protections. China’s extensivefacial recognition systems and social credit mechanisms demonstrate the dangers ofunchecked algorithmic governance.39
United States
The United States remains innovation-driven and comparatively reluctant to impose strict AIregulations. Private corporations dominate AI development, particularly withinSiliconValley.
The competition between the US and China increasingly resembles a technological ColdWarcentred upon AI supremacy.
ETHICAL LIMITATIONS OF AI IN JUDICIAL DECISION-MAKING
AI systems fundamentally differ from human reasoning. Human judgment involves empathy, conscience, ethical reflection, social understanding, proportionality, and contextual evaluation. AI lacks these qualities. It merely predicts outputs based on patterns withindata.40This distinction becomes especially important in constitutional adjudication where judgesfrequently confront moral dilemmas, conflicting rights, and competing constitutional values.
A human judge may temper strict legality with compassion. AI cannot. Moreover, excessivedependence upon AI creates “cognitive offloading,” where judges gradually surrenderanalytical skills to machines. Over time, judicial reasoning abilities may weaken.
The legal system therefore risks not only technological dependence but intellectual erosion. THE WAY FORWARD: AI AS AN ASSISTIVE TOOL
Despite these concerns, AI possesses undeniable benefits when used appropriately. AI maylegitimately assist with transcription, scheduling, legal research, translation, document organisation, administrative management, and spelling or grammatical correction. Thesefunctions reduce routine burdens and improve efficiency without compromising judicial independence.
However, substantive adjudicatory functions must remain exclusively human. Judges alonemust appreciate evidence, analyse facts, weigh competing arguments, exercise discretion, andDeliver reasoned conclusions. 41Several reforms are necessary.
1. National Judicial AI Oversight Committee
India should establish a specialised oversight body under the Supreme Court’s e-Committeeto regulate judicial AI usage.
2. Judicial Training
Judges require specialised training regarding AI limitations, algorithmic bias, andethical risks.
3. Explainable AI Standards
Any AI tools used within courts must satisfy strict transparency and explainabilityrequirements.
4. Legislative Framework
India urgently requires comprehensive AI legislation addressing accountability, privacy, transparency, and human rights protections. 42The proposed Artificial Intelligence Ethics andAccountability Bill, 2025 represents an important beginning, though it has not yet beenenacted.
CONCLUSION: JUSTICE CANNOT BE AUTOMATED
Artificial Intelligence represents one of the most transformative technological developmentsin human history. Within the legal system, it promises efficiency, accessibility, andadministrative reform. Yet the pursuit of efficiency cannot justify the erosionofconstitutional values.
Law is not mathematics. Judging is not optimisation. Courts do not merely processinformation; they interpret human suffering, balance competing rights, and protect constitutional morality. A human judge does more than identify legal patterns. Judgesunderstand context, dignity, vulnerability, and consequence. They possess conscience. AIsystems, however advanced, cannot replicate conscience. The future of AI in lawmust therefore remain carefully limited. AI may assist judicial administration, legal research, andprocedural efficiency. It may function as an intelligent clerical assistant. But it must neverreplace the application of the human mind.
In constitutional democracies, judicial accountability cannot be delegated to algorithms. Public confidence in the judiciary depends upon the belief that justice emerges fromhumanreasoning guided by constitutional morality rather than machine-generated probabilities. The“red line” must therefore remain clear: AI may support justice, but it cannot become justiceitself. Ultimately, the legitimacy of the legal system depends not upon speed or automation, but upon humanity. Courts exist not merely to decide cases but to uphold dignity, fairness, and constitutional conscience. Those responsibilities belong to human beings alone.