Task Decomposition for AI Automation

What Law Students Need to Know Before Building Anything

Adapted from materials developed for the AI Law and Policy Clinic, University of Michigan Law School.

I’ve been working with a group of law students at the University of Michigan this semester on AI projects, and I want to share something that has come out of that experience. This particular group has impressed me with both the creativity of their project ideas and the seriousness with which they approach the work. They are not interested in AI as a novelty. Their projects are oriented toward building capacity in under-resourced areas of legal practice, improving service quality for people who need it, and extending the reach of practitioners who are already doing more than their caseloads should allow. Ethical and responsible use is not an afterthought for this group; it is the starting point.

That commitment to purpose is also what makes the technical execution matter. When the goal is improving access to justice, the system has to work reliably. Which brings us to the question that follows every good project idea: where do you actually start?

The answer is not learning to code, nor is it picking the right model. It is learning to think clearly about what you want your system to do, and then breaking that vision down into pieces small enough to build, test, and trust. This process is called task decomposition, and it is the single most important skill in AI automation. Everything else follows from it.

What I’ve laid out below are seven principles that have emerged from my own experience building AI systems for research and legal applications. This area is still new and developing rapidly; there is no established playbook. These are lessons I have learned through iteration, and they continue to evolve as the technology and our understanding of it mature. I share them here as a starting framework that will need to be adapted as we all learn more.

Define Your Deliverable First

Before you write a single prompt or configure any tool, you need to describe precisely what your system will produce. Not what it will “do.” What will it produce? A deliverable is a concrete, observable output. If you cannot describe it in one sentence, you are not ready to build.

“A one-page summary of the key holdings and procedural history of a given appellate case” is a deliverable. “An AI that helps with legal research” is not. The difference matters because without a concrete target, you cannot evaluate whether your system works, scope the project, or communicate your goals to anyone else on your team.

One lesson I have learned the hard way is the pull toward over-featurizing. It is natural to envision a system that drafts summaries, extracts key terms, identifies related cases, and generates client communications. But a system that reliably does one thing well is far more valuable than one that attempts five things and does none of them consistently. Start narrow.

Establish Objective Success Criteria

Here is the question that every project needs to answer early: how do you know your system is working as expected?

It is tempting to rely on impressions. The output “seems good” or “looks right.” But AI systems produce outputs that can appear polished and plausible while being substantively wrong. In legal contexts, this is not just a quality issue; it is a reliability issue. A well-formatted case summary that omits the holding creates false confidence, and that is worse than having no summary at all.

You need measurable indicators. Can you compare the system’s output against a known-correct answer? Can you apply a consistent rubric? If not, the task may not be ready for automation. Build a set of test inputs with known correct outputs and run your system against them. Measure accuracy, completeness, and consistency. And pay attention to the distinction between correctness and quality. A grammatically polished paragraph that gets the law wrong is a failure, regardless of how professional it reads.

Build Lean, Then Expand

Do not attempt to build a full end-to-end solution from the start. Build the smallest version that solves a clearly defined problem, verify that it works, and only then add complexity.

This principle matters because complex systems fail in complex ways. When an end-to-end pipeline produces a bad output, diagnosing whether the error occurred at step two or step seven is genuinely difficult. A lean system with a narrow scope gives you a reliable foundation on which to build. If your ultimate goal is to automate client intake, begin by automating just the extraction of key facts from an intake form. Prove that it works before adding downstream steps.

There is also an important corollary here: not every step in a process needs to be automated. A system that reliably handles three of five steps and leaves two for human completion may deliver substantial value. Accepting partial automation is not a compromise. It is often the smartest design decision you can make.

Decompose the Task into Human-Defined Steps

Task decomposition is the process of breaking a complex task into discrete, sequential sub-tasks, each of which can be described, executed, and evaluated independently. This is the analytical core of any automation project.

Large language models perform best when given focused, well-specified instructions for a single operation. Asking a model to “analyze this contract and produce a risk assessment” combines multiple cognitive operations: reading, identifying relevant clauses, classifying risk levels, assessing severity, and generating a written assessment. Each of these is a distinct task with distinct failure modes. When you decompose the work, you can identify exactly where the system succeeds and where it breaks down.

The practical method is straightforward. Walk through the task as if you were training a new associate or paralegal who has never done this specific work before. Write down each discrete action: read the document, identify the parties, locate the indemnification clause, classify its scope, and so forth. If any step requires the performer to make multiple judgments simultaneously, break it down further. Then sequence the steps logically. Which steps depend on the output of a prior step? Which can be performed in parallel?

The key point here is that humans define the steps. The AI executes them. It does not determine what they should be.

Apply the Human Reliability Test

This principle provides a fundamental constraint on what can be automated: if a trained professional cannot perform a given step with high reliability and consistency, that step cannot be reliably automated.

The useful formulation is what I call the inter-rater reliability standard. If you cannot train two professionals, working independently, to arrive at the same output for the same input, the task is not ripe for automation. Consider the difference:

High reliability: “Extract the effective date from this contract.” Two trained paralegals will almost always identify the same date. This is an excellent candidate for automation.

Moderate reliability: “Classify this contract clause as either an indemnification provision or a limitation of liability.” Trained professionals will generally agree, though edge cases exist. Automation is possible with careful design.

Low reliability: “Assess whether this contract is favorable to our client.” This requires contextual judgment that varies across professionals depending on the client’s priorities, risk tolerance, and strategic objectives. This is not a candidate for automation.

Steps that fail the reliability test should not be discarded from the workflow. They should be designated as human decision points. Your automated system can prepare the information needed for that decision, organize it, and present it clearly. The judgment itself remains with a human professional.

Write the Process as a Manual Procedure

Before automating anything, write a complete procedural description of the task. Write it at the level of specificity you would use to train a competent professional who has never performed this particular work before.

Writing forces precision. Ambiguities that are invisible when you “just know” how to do something become immediately apparent when you must explain every decision point on paper. These ambiguities are exactly the points where an automated system will fail unpredictably.

Document inputs and outputs for each step. Specify decision rules explicitly: when you encounter X, do Y; when the clause contains language Z, classify it as category A; if none of the categories apply, flag for human review. Identify exceptions and edge cases. What happens when the document is missing expected information? When is the input ambiguous? When two rules conflict?

Here is the critical test: have someone else follow your written instructions on a sample input. If they produce an output different from what you expected, the procedure needs refinement. The problem is with the instructions, not the person.

Evaluate Each Sub-Task Independently

Once you have decomposed the task and written the procedure, each sub-task must be tested independently before being integrated into the larger pipeline.

The reason is mathematical. If step one is 90% accurate and step two is 90% accurate, the combined accuracy is not 90%. It is 81%. Add a third step at 90%, and you are down to 73%. Errors compound through a pipeline, and the only way to manage this is to understand the reliability of each component individually. This understanding allows you to make informed decisions about where to invest improvement effort and where to insert human checkpoints.

Create test cases for each sub-task, including typical inputs, edge cases, and adversarial inputs where you expect the system to struggle. Measure performance quantitatively. Set minimum performance thresholds. If a sub-task cannot meet its threshold, it needs redesign, additional constraints, or human oversight before it joins the pipeline.

---

These seven principles are not sequential prerequisites that you complete once and move past. They are iterative and interdependent. As you build and test, you will return to earlier principles to refine your deliverable definition, adjust your decomposition, or tighten your success criteria. This iterative refinement is a natural and productive part of the process, and it mirrors my own experience; I still revisit these principles with every new project.

This is a young field, and all of us are learning as we go. The principles above represent my best current thinking, shaped by what has worked and what has not in my own projects. They will continue to evolve. The ideas that this group of students is developing deserve that kind of structured foundation, and I look forward to seeing where they take them.