AI Literacy for Legal AI Systems: a Practical Approach
Abstract
Legal AI systems are increasingly being adopted by judicial and legal system deployers and providers worldwide to support a range of applications. While they offer potential benefits such as reducing bias, increasing efficiency, and improving accountability, they also pose significant risks, requiring a careful balance between opportunities, and legal and ethical development and deployment. AI literacy, as a legal requirement under the EU AI Act and a critical enabler of ethical AI for deployers and providers, could be a tool to achieve this. The article analyzes the notion of AI literacy, the benefits and risks of legal AI systems, and links them to a broader ethical legal AI system for organizations. The outcome of the article, a roadmap questionnaire as a practical tool for developers and providers to assess risks, benefits, and stakeholder concerns, could be useful in meeting societal and regulatory expectations for legal AI systems.
References
in charts. McKinsey Global Institute, 2024. https://www.mckinsey.com/mgi/our-research/mckinsey-global-institute-2024-in-charts
AI Literacy – Questions & Answers. European Commission, 2025. https://digital-strategy.ec.europa.eu/en/faqs/ai-literacy-questions-answers
Alnemr, Nardine: Democratic self-government and the algocratic shortcut: The democratic harms in algorithmic governance of society. Contemporary Political Theory, Vol. 23. (2024) https://doi.org/10.1057/s41296-023-00656-y
Alpaydin, Ethem: Machine learning: The new AI. MIT Press, 2016.
Annapureddy, Ravinithesh – Fornaroli, Alessandro – Gatica-Perez, Daniel: Generative AI literacy: Twelve defining competencies. Digital Government: Research and Practice, Vol. 6., N. 1. (2024) https://doi.org/10.1145/3685680
Annual report 2023, ECtHR. https://www.echr.coe.int/documents/d/echr/annual-report-2023-eng
Barry, Brian M.: How judges judge: Empirical insights into judicial decision-making. Routledge, 2021. https://doi.org/10.4324/9780429023422
Beckers, Anna – Teubner, Gunther: Human–algorithm hybrids as (quasi-) organizations? On the accountability of digital collective actors. Journal of Law and Society, Vol. 50., N. 1. (2024) https://doi.org/10.1111/jols.12412
Beijing consensus on artificial intelligence and education. UNESCO, 2019. https://unesdoc.unesco.org/ark:/48223/pf0000368303.
Björklund, Fredrika: Trust and surveillance: An odd couple or a perfect pair? In: Lora Anne Viola – Paweł Laidler (eds.): Trust and transparency in an age of surveillance. Routledge, 2021. https://doi.org/10.4324/9781003120827
Bozdağ, Mustafa – Sevim, Nurullah – Koç, Aykut: Measuring and mitigating gender bias in legal contextualized language models. ACM Transactions on Knowledge Discovery from Data, Vol. 18., N. 4. (2024) https://doi.org/10.1145/3628602
Brożek, Bartosz et al.: The black box problem revisited: Real and imaginary challenges for automated legal decision-making. Artificial Intelligence and Law, Vol. 32. (2024) https://doi.org/10.1007/s10506-023-09356-9
Burrell, Jenna: How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society, Vol. 3., N. 1. (2016) https://doi.org/10.1177/2053951715622512
Casal-Otero, Lorena et al.: AI literacy in K-12: A systematic literature review. International Journal of STEM Education, Vol. 10. (2023) 29. https://doi.org/10.1186/s40594-023-00418-7
Ng, Davy Tsz Kit et al.: Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, Vol. 2. (2021)100041. https://doi.org/10.1016/j.caeai.2021.100041
Černý, Michal: AI literacy in Higher Education: Theory and Design. In: Ł. Tomczyk (ed.): New Media Pedagogy: Research Trends, Methodological Challenges, and Successful İmplementations. Communications in Computer and Information Science. Springer, 2024. https://doi.org/10.1007/978-3-031-63235-8_24
Migliorini, Sara – Moreira J. I., João Ilhão: The case for nurturing AI literacy in law schools. Asian Journal of Legal Education, Vol. 11., N. 1. (2024) https://doi.org/10.1177/23220058241265613
Castelliano, Caio – Grajzl, Peter – Watanabe, Eduardo: Does electronic case-processing enhance court efficacy? New quantitative evidence. Government Information Quarterly, Vol. 40., N. 4. (2023) 101861. https://doi.org/10.1016/j.giq.2023.101861
Çelebi, Celalettin et al.: Artificial intelligence literacy: An adaptation study. Instructional Technology and Lifelong Learning, Vol. 4., N. 2. (2023) 291–306. https://doi.org/10.52911/itall.1401740
Çelik, Ismail: Exploring the determinants of artificial intelligence (AI) literacy: Digital divide, computational thinking, cognitive absorption. Telematics and Informatics, Vol. 83. (2023) 102026. https://doi.org/10.1016/j.tele.2023.102026
CEPEJ: Resource Centre Cyberjustice and AI Jun 6, 2025. https://tinyurl.com/mr3h8j8h
Çetindamar, Dilek et al.: Explicating AI literacy of employees at digital workplaces. IEEE Transactions on Engineering Management, Vol. 71., N. 3. (2024) https://doi.org/10.1109/TEM.2021.3138503
Chromik, Michael et al.: I think I get your point, AI! The illusion of explanatory depth in explainable AI. In: Proceedings of the 26th International Conference on Intelligent User Interfaces (IUI ’21), New York, ACM, 2021. https://doi.org/10.1145/3397481.3450644
Cox, Andrew: Algorithmic literacy, AI literacy, and responsible generative AI literacy. Journal of Web Librarianship, Vol. 18., N. 3. (2024) https://doi.org/10.1080/19322909.2024.2395341
Cui, Yadong: Artificial intelligence and judicial modernization. Springer Nature Singapore Pte Ltd, 2020. https://link.springer.com/book/10.1007/978-981-32-9880-4
Decoding responsibility in the era of automated decisions: Understanding the implications of the CJEU's SCHUFA judgment. Centre for Information Policy Leadership, 2024. https://tinyurl.com/4van5x3x
Deroy, Aniket – Maity, Subhankar: Questioning biases in case judgment summaries: Legal datasets or large language models? arXiv Preprint. https://doi.org/10.48550/arXiv.2312.00554
Ebers, Martin: Truly Risk-Based Regulation of Artificial Intelligence How to Implement the EU’s AI Act. European Journal of Risk Regulation, 2024. https://doi.org/10.1017/err.2024.78
Farah, Hibaq: Court of appeal judge praises ‘jolly useful’ ChatGPT after asking it for legal summary. The Guardian, 15 September 2023. https://tinyurl.com/bdfd7jm8 ; Molly Bohannon: Lawyer Used ChatGPT In Court And Cited Fake Cases: A Judge Is Considering Sanctions. Forbes, 8 June 2023. https://tinyurl.com/5axdnd8n
Figaredo, Daniel Domínguez– Julia Stoyanovich: Responsible AI literacy: A stakeholder-first approach. Big Data & Society, Vol. 10., N. 2. (2023). https://doi.org/10.1177/20539517231219958
Fitsilis, Fotis – Mikros, George: AI-based solutions for legislative drafting in the EU Summary report. European Commission Directorate-General for Digital Services, 2024. https://tinyurl.com/3y7ztkkk
Fjelstul, Joshua C. – Gabel, Matthew – Carrubba, Clifford J.: The timely administration of justice: Using computational simulations to evaluate institutional reforms at the CJEU. Journal of European Public Policy, Vol. 30., N. 12. (2022) https://doi.org/10.1080/13501763.2022.2113115
Fomin, Vladislav V. – Astromskis, Paulius: The Black Box Problem. In: John-Stewart Gordon (ed.): Future Law, Ethics, and Smart Technologies. Leiden, Brill, 2023. https://doi.org/10.1163/9789004682900_012
Forrest, Katherine B.: When machines can be judge, jury, and executioner: Justice in the age of artificial intelligence. World Scientific Publishing Company, 2021. https://doi.org/10.1142/12172
Francesconi, Enrico: The winter, the summer, and the summer dream of artificial intelligence in law. Artificial Intelligence and Law, Vol. 30. (2022) https://doi.org/10.1007/s10506-022-09309-8
Ghasemaghaei, Maryam – Kordzadeh, Nima: Understanding how algorithmic injustice leads to making discriminatory decisions: An obedience to authority perspective. Information & Management, Vol. 61., N. 2. (2024) 103921. https://doi.org/10.1016/j.im.2024.103921
Gravett, W. H.: Is the dawn of the robot lawyer upon us? The fourth industrial revolution and the future of lawyers. Potchefstroom Electronic Law Journal, Vol. 23. (2020) https://scholarship.law.duke.edu/dlj/vol68/iss6/2 https://doi.org/10.17159/1727-3781/2020/v23i0a6794
Guidelines and Regulations to Provide İnsights on Public Policies to Ensure AI’s Beneficial Use as a Professional Tool. International Bar Association, 18 September, 2024. https://www.ibanet.org/PPID/Constituent/Multi-displry_Pract/anlbs-ai-report
Gutiérrez, Juan David: ChatGPT in Colombian Courts: “Why we need to have a conversation about the digital literacy of the judiciary”. Verfassunblog, 23 February 2023. https://verfassungsblog.de/colombian-chatgpt/
Hacker, Philipp: Sustainable AI regulation. Common Market Law Review, Vol. 61., N. 2. (2024) https://doi.org/10.54648/cola2024025
Hamon, Ronan et al.: Bridging the gap between AI and explainability in the GDPR: Towards trustworthiness-by-design in automated decision-making. IEEE Computational Intelligence Magazine, Vol. 17., N. 1. (2022) https://doi.org/10.1109/MCI.2021.3129960
Harper, Christopher: Using GPT-4 to generate 100 words consumes up to 3 bottles of water — AI data centers also raise power and water bills for nearby residents. Tom’s Hardware, 19 September 2024. https://tinyurl.com/mr34dx58
Joshi, Abhinav et al.: IL-TUR: Benchmark for Indian legal text understanding and reasoning. arXiv Preprint, (2024). https://arxiv.org/abs/2407.05399
Katz, Daniel Martin – Bommarito, Michael J. – Blackman, Josh: A general approach for predicting the behavior of the Supreme Court of the United States. PLOS ONE Vol. 12., N. 4. (2017). https://doi.org/10.1371/journal.pone.0174698
Kelemen, Katalin – de Miranda, Luis: Courts as anthrobots: Learning from human forms of interaction to develop a philosophically healthy model for judicial automation. International Journal for Court Administration, Vol. 15., N. 2. (2024) 2. https://doi.org/10.36745/ijca.525
Kinger, Shakti – Kulkarni, Vrushali: Demystifying the black box: An overview of explainability methods in machine learning. International Journal of Computers and Applications, Vol. 46., N. 2. (2023) https://doi.org/10.1080/1206212X.2023.2285533
Knoth, Nils et al.: Developing a holistic AI literacy assessment matrix: Bridging generic, domain-specific, and ethical competencies. Computers and Education Open, Vol. 6. (2024) 100177. https://doi.org/10.1016/j.caeo.2024.100177
Lai, Jinqi et al.: Large language models in law: A survey. (5). AI Open, Vol. 5. (2024) https://doi.org/10.1016/j.aiopen.2024.09.002
LaRosa, Emily – Danks, David: Impacts on trust of healthcare AI. In: ACM Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. 2018. https://doi.org/10.1145/3278721.3278771
Laupichler, Matthias Carl et al.: Development of the “Scale for the assessment of non-experts’ AI literacy” – An exploratory factor analysis. Computers in Human Behavior Reports, Vol. 12. (2023) 100338. https://doi.org/10.1016/j.chbr.2023.100338
Lin, Zhiyuan Jerry et al.: The limits of human predictions of recidivism. Science Advances, Vol. 6., N. 1. (2020) https://doi.org/10.1126/sciadv.aaz0652
Linardatos, Pantelis – Papastefanopoulos, Vasilis – Kotsiantis, Sotiris: Explainable AI: A review of machine learning interpretability methods. Entropy, Vol. 23., N. 1. (2021) https://doi.org/10.3390/e23010018
Living repository to foster learning and exchange on AI literacy. European Commission, 4 February 2025. https://tinyurl.com/2hah9evj
Long, Brandon – Palmer, Amitabha: AI and access to justice: How AI legal advisors can reduce economic and shame-based barriers to justice. TATuP – Zeitschrift für Technikfolgenabschätzung in Theorie Und Praxis, Vol. 33., N. 1. (2024) https://doi.org/10.14512/tatup.33.1.21
Long, Duri – Magerko, Brian: What is AI literacy? Competencies and design considerations. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. New York, ACM, (2020). https://doi.org/10.1145/3313831.3376727
Lütz, Fabian: The AI Act, gender equality, and non-discrimination: What role for the AI office? ERA Forum, Vol. 25. (2024) https://doi.org/10.1007/s12027-024-00785-w
Markovic, Milan: Rise of the robot lawyers. Arizona Law Review, Vol. 61., No. 2. (2019) 325–350. https://arizonalawreview.org/pdf/61-2/61arizlrev325.pdf
Maslej, Nestor et al.: The AI Index 2024 Annual Report. AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, (2024) https://doi.org/10.48550/arXiv.2405.19522
Matthias, Andreas: The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology, Vol. 6., N. 3. (2004) https://doi.org/10.1007/s10676-004-3422-1
Medvedeva, Masha – Wieling, Martijn – Vols, Michel: Rethinking the field of automatic prediction of court decision. Artificial Intelligence and Law, Vol. 31. (2023) https://doi.org/10.1007/s10506-021-09306-3
Millar, Jason – Kerr, Ian: Delegation, relinquishment, and responsibility: The prospect of expert robots. In: Ryan Calo – A. Michael Froomkin – Ian Kerr (eds.): Robot Law. Cheltenham, Edward Elgar Publishing. 2016. https://doi.org/10.4337/9781783476732.00012
Moran, Lyle: Lawyer cites fake cases generated by ChatGPT in legal brief. LegalDive, 30 May 2023. https://tinyurl.com/bdcxxwkj
Mori, Masahiro: The uncanny valley. (1970). The original essay by Masahiro Mori (K. F. MacDorman & N. Kageki, Trans.). IEEE Spectrum, 12 Jun 2012. https://spectrum.ieee.org/automaton/robotics/humanoids/the-uncanny-valley https://doi.org/10.1109/MRA.2012.2192811
Myers, Andrew: Rooting out anti-Muslim bias in popular language model GPT-3. Stanford HAI, 3 July 2022. https://hai.stanford.edu/news/rooting-out-anti-muslim-bias-popular-language-model-gpt-3
Nass, Clifford – Moon, Youngme: Machines and mindlessness: Social responses to computers. Journal of Social Issues, Vol. 56., N. 1. (2000) https://doi.org/10.1111/0022-4537.00153
Nass, Clifford – Steuer, Jonathan – Tauber, Ellen R.: Computers are social actors. In: ACM Proceedings of SIGCHI ’94 Human Factors in Computing Systems. 1994. https://doi.org/10.1145/259963.260288
O’Hara, Matthew J.: I, for one, welcome our new AI jurors: ChatGPT and the future of the jury system in American law. International Journal of Law, Ethics, and Technology, Vol. 3., N. 2. (2024) https://www.ijlet.org/4.3.2 https://doi.org/10.55574/ILSM6729
Pavlidis, Georgios: Unlocking the black box: Analysing the EU artificial intelligence act’s framework for explainability in AI. Law, Innovation and Technology, Vol. 16., N. 1. (2024) https://doi.org/10.1080/17579961.2024.2313795
Pinski, Marc – Benlian, Alexander : AI literacy for users – A comprehensive review and future research directions of learning methods, components, and effects. Computers in Human Behavior: Artificial Humans, Vol. 2., N. 1. (2024) 100062. https://doi.org/10.1016/j.chbah.2024.100062
Portela, Manuel et al.: A comparative user study of human predictions in algorithm-supported recidivism risk assessment. Artificial Intelligence and Law, (2024). https://doi.org/10.1007/s10506-024-09393-y
Raymond, Nate: US judge runs 'mini-experiment' with AI to help decide case. Reuters, 6 September 2024. https://tinyurl.com/pfjr6aab
Re, Richard M. – Solow-Niederman, Alicia: Developing artificially intelligent justice. Stanford Technology Law Review, Vol. 22., N. 2. (2019) https://www.law.virginia.edu/scholarship/publication/richard-m-re/923611
Rebstadt, Jonas et al.: Towards personalized explanations for AI systems: Designing a role model for explainable AI in auditing. Wirtschaftsinformatik Proceedings, Vol. 2. (2022) https://aisel.aisnet.org/wi2022/ai/ai/2
Reed, Chris – Grieman, Keri – Early, Joseph: Non-Asimov explanations: Regulating AI through transparency. Nordic Yearbook of Law and Informatics 2020–2021: Law in the Era of Artificial Intelligence, mars 2022 (2022) https://doi.org/10.53292/208f5901.20b0a4e7
Saunders, Tom: Legal tech teams turn to AI to advance business goals. Financial Times, 19 October 2023. https://www.ft.com/content/9a117ac7-29ae-43fe-b840-a04005b98799
Segal, Eddie: The impact of AI on cybersecurity. IEEE Computer Society, 31 March 2024. https://tinyurl.com/4sya5rh6
Shi, Changqing et al.: The smart court – A new pathway to justice in China? International Journal for Court Administration, Vol. 12., N. 1. (2021) https://doi.org/10.36745/ijca.367
Shneiderman, Ben: Bridging the gap between ethics and practice: Guidelines for reliable, safe, and trustworthy human-centered AI systems. ACM Transactions on Interactive Intelligent Systems, Vol. 10., N. 4. (2020) https://doi.org/10.1145/3419764
Steenhuis, Quinten – Westermann, Hannes: Getting in the door: Streamlining intake in civil legal services with large language models. arXiv Preprint, (2024) https://arxiv.org/abs/2410.03762 https://doi.org/10.3233/FAIA241242
Susskind, Richard: Online Courts and the Future of Justice. New York, Oxford Academic, 2019. (online ed.) https://doi.org/10.1093/oso/9780198838364.001.0001
Susskind, Richard: Tomorrow’s lawyers: An introduction to your future. Oxford, Oxford University Press, 2013.
Taddy, Matt: The technological elements of artificial intelligence. In: Ajay Agrawal – Joshua Gans – Avi Goldfarb (eds.): The economics of artificial intelligence: An agenda. National Bureau of Economic Research – University of Chicago Press, 2019.
Tamošiūnienė, Egidija – Terebeiza, Žilvinas – Doržinkevič, Artur: The possibility of applying artificial intelligence in the delivery of justice by courts. 17(1), Baltic Journal of Law & Politics, Vol. 17., N. 1. (2024) https://doi.org/10.2478/bjlp-2024-0010
Tevis, Walter: Mockingbird. RosettaBooks LLC, 1980.
The future is now: Artificial intelligence and the legal profession. International Bar Association, 2024. https://tinyurl.com/6x3twwxj
Third AI Pact webinar on AI literacy. 20 February, 2025. https://www.youtube.com/watch?v=Dyf4ZVts9HY
Valentin, Sarah: Impoverished algorithms: Misguided governments, flawed technologies, and social control. Fordham Urban Law Journal, Vol. 46., N. 2. (2019) https://ir.lawnet.fordham.edu/ulj/vol46/iss2/4/
Van Wynsberghe, Aimee: Artificial intelligence: From ethics to policy. European Parliament Panel for the Future of Science and Technology, 2020. https://tinyurl.com/macpukkz
Vasdani, Tara: Robot justice: China’s use of Internet courts. The Lawyer’s Daily, 5 February 2020. https://www.thelawyersdaily.ca/articles/17741/robot-justice-china-s-use-of-internet-courts
Veron, Mathieu: Do this LLMs use my prompt data for training. Medium, 2 July 2024. https://tinyurl.com/587346s3
Volokh, Eugene: Chief justice robots. Duke Law Journal, Vol. 68., N. 6. (2019)
Wachter, Sandra – Mittelstadt, Brent – Russell, Chris: Why fairness cannot be automated: Bridging the gap between EU non-discrimination law and AI. Computer Law & Security Review, Vol. 41. (2021) 105567. https://doi.org/10.1016/j.clsr.2021.105567
Wachter, Sandra: Limitations and loopholes in the EU AI Act and AI liability directives: What this means for the European Union, the United States, and beyond. Yale Journal of Law and Technology, Vol. 26., N. 3. (2024) http://dx.doi.org/10.2139/ssrn.4924553
Wang, Bingcheng – Rau, Pei-Luen Patrick – Yuan, Tianyi: Measuring user competence in using artificial intelligence: Validity and reliability of artificial intelligence literacy scale. Behaviour & Information Technology, Vol. 42., N. 9. (2022) https://doi.org/10.1080/0144929X.2022.2072768
Winter, Christoph K.: The challenges of artificial judicial decision-making for liberal democracy. In: P. Bystranowski – P. Janik – M. Próchnicki (eds.): Judicial decision-making: Integrating empirical and theoretical perspectives. (Economic Analysis of Law in European Legal Scholarship, Vol. 14.) Cham, Springer, 2021. https://doi.org/10.1007/978-3-031-11744-2_9
Without universal AI literacy, AI will fail us. World Economic Forum, Marc 17, 2022. https://www.weforum.org/stories/2022/03/without-universal-ai-literacy-ai-will-fail-us/
Yeung, Karen – Harkens, Adam: How do ‘technical’ design-choices made when building algorithmic decision-making tools for criminal justice authorities create constitutional dangers? Part II. Public Law, (April, 2023) https://tinyurl.com/ycy39trr
Copyright (c) 2025 Gizem Gültekin-Várkonyi
This work is licensed under a Creative Commons Attribution 4.0 International License.
