The LLM Assistant

While manual coding is the foundation of qualitative discourse analysis, it can become incredibly time-consuming when dealing with massive datasets. To accelerate your research workflow, DATS features an integrated LLM Assistant that leverages state-of-the-art Large Language Models (LLMs) to automatically suggest tags, extract metadata, and generate text annotations.

Because the assistant operates strictly within DATS using secure, locally hosted models (like Gemma or Llama), your sensitive research data remains entirely private.

(For a deeper dive into the methodology and evaluation of this AI-assisted workflow, please refer to our publications: Exploring Large Language Models for Qualitative Data Analysis and Semi-automatic Sequential Sentence Classification in the Discourse Analysis Tool Suite).

1. Accessing the Assistant

Before the AI can help you, it needs to know which documents to analyze. Therefore, you can access the LLM Assistant in two different ways depending on your current workflow:

Method A: Single Document (From Annotation View)

When you are deep-reading a specific file in the Annotation View, you can ask the assistant to help you code that exact file.

1. Look at the top Annotation Toolbar (where you select your reading mode).
2. Click the Robot icon.
3. This opens the LLM Assistant Dialog to process only the currently open document.

Method B: Batch Processing (From Search View)

If you want the AI to analyze dozens or hundreds of documents at once, you must initiate the job from the Search View.

1. Use the search bar and filters to find the documents you want to analyze.
2. Select the documents using the checkboxes in the results table. (Tip: Use the master checkbox at the top to select all visible rows).
3. Once at least one document is selected, the Robot icon will appear in the top toolbar. Click it to open the dialog for your selected batch.

2. The Multi-Step Workflow

The dialog guides you through a simple, step-by-step configuration process. (The following example assumes you chose Span Annotation).

1. Choosing a Task

Select the specific qualitative task you want the AI to automate.

When the dialog opens, the assistant will greet you and ask what kind of help you need. You can choose from four distinct AI tasks, which guide you through the following steps:

• Document Tagging: The AI will read the documents and suggest which structural Tags (e.g., Domain: Politics, Sentiment: Negative) should be applied to them as a whole.
• Metadata Extraction: The AI will scan the text to automatically populate custom metadata fields (e.g., finding the Author or Publication Date hidden in the text).
• Span Annotation: The AI will highlight specific words or phrases and assign them to your qualitative Codes.
• Sentence Annotation: The AI will classify entire, discrete sentences according to your Codebook.

Or choose Prompt to run one of your prompts. A prompt decides everything except the documents and the model: the task, the codes, tags or metadata fields, the few-shot examples and the input and output format. You select the prompt and its version, then the model, review the prompt and start. Write prompts, versions and examples on the Prompts page.

Step 2: Select Targets (Codes, Tags, or Metadata)

You must tell the AI what to look for.

• A list of your project's Codes (or Tags/Metadata) will appear.
• Use the checkboxes to select the specific categories you want the AI to apply.
• Note: The AI relies heavily on the Descriptions you wrote for these codes in the Project Settings. The better your written description, the more accurate the AI's predictions will be!

Step 3: Edit Settings

Choose the language model and how the assistant works:

• Language of the prompt (English or German). The documents can be in any language.
• Input format: send each document as a whole, sentence by sentence, or in overlapping chunks. Tagging and metadata extraction always send the whole document. Chunks have input parameters: chunk size and overlap.
• Output format, i.e. how the model answers and how DATS maps the answers back to your documents. For span annotation: inline tags (the model repeats the text with tagged passages) or quotes with context (the model returns quotes that DATS locates in the text). About ... explains the selected format with an example. Quotes with context have output parameters: the context length and the fuzzy matching threshold.

Step 4: Review the Prompt

The preview shows exactly what the AI receives: the built-in default prompt of the task, output format and language with your selections. Built-in prompts cannot be edited; every project has a copy of each on the Prompts page (e.g. default-span-annotation-inline-tagging-en), which you can edit and run with Prompt. Pick one of the selected documents; a prompt sent sentence by sentence can step through its sentences. Guided runs are zero-shot; for few-shot examples and your own prompt texts, write a prompt and run it with Prompt.

For span and sentence annotation you also choose whom the assistant annotates as: an existing virtual user of the project, or a new one you name here (proposed: the next free LLM-Run<n>). The assistant never writes into your own annotations. Use a new virtual user per run to compare runs, models or prompts with each other and with your team in the compare view and the agreement evaluation.

If the selected virtual user already annotated the selected documents with the selected codes, you choose here whether these annotations are replaced or kept.

Step 5: Wait (Background Processing)

Click Start! The dialog will close, and the job will run asynchronously in the background. You are free to continue reading or analyzing other documents while the AI works.

• To check the progress, simply click the Robot icon again to open the dialog and look at the Previous LLM Assistant Jobs list at the bottom of the welcome screen.

6. Reviewing and Applying Results

Because qualitative research requires human oversight, DATS never applies AI annotations directly to your project without your permission.

1. Once a job is finished, open the LLM Assistant dialog and click View Results next to the completed job.
2. You will be presented with a review screen showing the document text with all of the AI's suggestions highlighted.
3. Human-in-the-Loop: You can click on any AI suggestion to modify it. You can delete an incorrect suggestion or change its assigned code.
4. Once you are satisfied with the AI's work, click Apply Annotations!.

(Note: AI-generated annotations belong to the virtual user you chose, so they are always distinguishable from your own manual coding in the analysis views!)