This repository is the code base for the paper "Improving Argument Effectiveness Across Ideologies using Instruction-tuned Large Language Models" Accepted at EMNLP2024 Findings.
-
Generated effective arguments: [link]
- Format:
{ideology}_{model-type}_{n}shot{_steered}.jsonl(for example: conservative_chatgpt_0shot.jsonl)- 86BB ideology: refers to the target reader's ideology - it has one of these values: liberal or conservative
- model-type: refers to the Instruction-tuned LLM used: either chatgpt or llamav2 referring to Llama-2-7b-chat-hf
- n: refers to Zero- or One-shot prompting with values 0 or 1 resepctively
-
_steered: Empty if the LLM is not steered; otherwise it has the value of steered_meanl0.5 and steered_meanl0.2 where 0.5 and 0.2 refers to the
$lambda$ value
- Format:
- Dismissed arguments list: [link]
- Evaluation:[link]
Install the required packages from the requirements file using pip install -r requirements.txt.
The code is a Python package that can be downloaded locally with the following command when you access the code folder locally:
pip install .
The main folder contains the following:
requirements.txt: contains all dependenciespyproject.toml: contains the Python package information (e.g., version, dependencies, metadata)LICENSE: MIT licenseiesta: the python package that contains the main code for the paper (see below for more details)data: contains the original and generated datascriptsandnotebooks: execute theiestacode
- Paper Section 3. Data and Appendix A:
package:iesta.data
- Paper Section 5.1 Experiments:
package:iesta.llms, iesta.evaluator - Paper Section 5.2 Experiments POSTPROCESSING:
iesta.evaluator.generation_processor.py[link]