8000
Skip to content

Latest commit

 

History

12 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Provenance Guard

Provenance Guard is a Flask backend that helps creative platforms label text submissions with attribution context. It classifies writing with multiple signals, returns a confidence score, stores structured audit evidence, rate limits submissions, and lets creators appeal a decision.

Setup

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Create .env in the project root:

GROQ_API_KEY=your_key_here

.env is ignored by git. If GROQ_API_KEY is missing, the app still runs with a clearly flagged local development fallback for the LLM signal. Real demos and grading should use Groq so the first signal is an actual model-based classifier.

Run the app:

flask --app app:create_app run

Run tests:

python -m unittest

The default SQLite database is stored under Flask's instance/ directory.

Current Milestone 5 And Stretch Checkpoint

The current source implements the Milestone 5 production layer plus all four stretch features:

  • POST /submit
  • first detection signal: Groq LLM when GROQ_API_KEY is set, with a local fallback for offline development
  • second detection signal: stylometric heuristics
  • third detection signal: lexical pattern detection
  • ensemble confidence scoring using 50% Groq + 30% stylometry + 20% lexical patterns
  • structured SQLite audit logging
  • durable submission status storage
  • POST /appeal and POST /appeals appeal routes
  • per-IP submission rate limiting: 10 per minute and 100 per day
  • provenance certificate request, approval, lookup, and verified-human badge
  • content_type: "image_metadata" submissions
  • browser UI at GET /ui for text, text-file, image-metadata submissions, score display, appeals, and dashboard access
  • GET /dashboard analytics view with detection pattern, appeal rate, average confidence, content types, recent submissions, and appeals
  • GET /log

The dashboard and certificate routes are intentionally lightweight so they remain easy to demo and inspect for the course rubric.

Milestone 5 curl test:

curl -s -X POST http://localhost:5000/submit \
  -H "Content-Type: application/json" \
  -d '{"text": "The sun dipped below the horizon, painting the sky in hues of amber and rose. I sat on the porch, coffee in hand, watching the neighborhood slowly go quiet.", "creator_id": "test-user-1"}' | python -m json.tool

Expected response fields include content_id, attribution, confidence, confidence_score, ai_probability, label, content_type, metadata, ensemble, creator_certificate, verified_human_badge, signal_1, signal_2, signal_3, and signals.lexical_patterns. The same content_id is written to the audit log with llm_score, stylometry_score, lexical_pattern_score, label, status, content_type, metadata, and appeal_filed.

Appeal test:

curl -s -X POST http://localhost:5000/appeal \
  -H "Content-Type: application/json" \
  -d '{"content_id": "PASTE-CONTENT-ID-HERE", "creator_reasoning": "I wrote this myself from personal experience. I am a non-native English speaker and my writing style may appear more formal than typical."}' | python -m json.tool

POST /appeals is kept as a compatibility alias. The appeal response returns an appeal_id, the content_id, status: "under_review", the creator reasoning, and the original classification summary. The audit log receives a structured appeal_submitted event with the original signal payload.

Stretch demo flow:

curl -s -X POST http://localhost:5000/certificates/request \
  -H "Content-Type: application/json" \
  -d '{"creator_id": "creator-123", "verification_method": "portfolio_review", "evidence": {"portfolio_url": "https://example.com/creator-123"}}' | python -m json.tool

curl -s -X POST http://localhost:5000/certificates/PASTE-REQUEST-ID-HERE/approve \
  -H "Content-Type: application/json" \
  -d '{"reviewer_notes": "Draft history and portfolio reviewed."}' | python -m json.tool

curl -s -X POST http://localhost:5000/submit \
  -H "Content-Type: application/json" \
  -d '{"creator_id": "creator-123", "content": "This is a certificate-aware submission after manual verification."}' | python -m json.tool

curl -s -X POST http://localhost:5000/submit \
  -H "Content-Type: application/json" \
  -d '{"creator_id": "artist-88", "content_type": "image_metadata", "content": "A hand-drawn ink illustration of a city street at dusk.", "metadata": {"declared_creation_method": "hand drawn, scanned, color corrected", "alt_text": "Ink drawing of storefronts under evening light"}}' | python -m json.tool

Open http://127.0.0.1:5000/dashboard after generating a few submissions to see the analytics dashboard.

Browser UI:

http://127.0.0.1:5000/ui

The UI can score pasted text, read text files into the form, submit image captions and metadata, show individual signal scores, submit appeals, and link to the dashboard. It does not upload or inspect raw image pixels.

Audit log demo:

curl -s http://localhost:5000/log | python -m json.tool

Four-input scoring check used for Milestone 5 calibration:

Input type Groq/local score Stylometry score Lexical score Combined AI probability Result
Clearly AI-generated 0.9263 0.7794 1.0000 0.8970 likely_ai
Clearly human-written 0.0000 0.2212 0.0000 0.0664 likely_human
Borderline formal human writing 0.4567 0.7796 0.0000 0.4982 uncertain
Borderline lightly edited AI output 0.7305 0.5174 0.4333 0.6071 uncertain

Rate-limit evidence from the Flask test client, using 11 rapid requests from the same IP:

201
201
201
201
201
201
201
201
201
201
429

Architecture Narrative

A submitted piece of work enters through POST /submit with creator_id, optional title, content, optional content_type, and optional metadata. Flask validates the JSON body, checks required fields, and Flask-Limiter applies the per-IP submission limits. The default content_type is text; the stretch multi-modal path also supports image_metadata, where content is the image description or caption and metadata stores structured context.

The validated content goes to the detection pipeline. The required baseline uses the Groq LLM signal and stylometric signal. The stretch ensemble adds a lexical pattern signal, so each submission can return three signal scores plus the documented ensemble weights. The scoring layer applies the classification thresholds and builds the transparency label. SQLite stores the submission, signal results, certificate context, and audit event before the API returns the structured response to the platform.

If a creator contests the result, POST /appeal or the POST /appeals alias looks up the original submission, records the creator's reason, updates the submission status to under_review, and logs the appeal with the original decision.

Component Organization

The code can be refactored into smaller components for readability. The intended ownership is:

File Components
app.py Minimal Flask entrypoint.
provenance_guard/__init__.py App factory, route registration, database initialization, and limiter setup.
provenance_guard/config.py Model names, rate limits, database path, thresholds, and signal weights.
provenance_guard/db.py SQLite connection, schema, and database helpers.
provenance_guard/labels.py Transparency label templates and label builder.
provenance_guard/scoring.py Signal weighting, confidence calculation, and result thresholds.
provenance_guard/audit.py Audit event creation and serialization.
provenance_guard/certificates.py Verification requests, manual approvals, certificate lookup, and verified-human badge text.
provenance_guard/analytics.py Dashboard metrics for detection counts, appeal rate, and average confidence.
provenance_guard/modalities.py Content type validation and normalization for text and image_metadata.
provenance_guard/validation.py Request validation, normalization, and JSON error helpers.
provenance_guard/routes/submissions.py POST /submit and root service metadata endpoint.
provenance_guard/routes/appeals.py POST /appeal, POST /appeals, and appeal status updates.
provenance_guard/routes/logs.py GET /log audit-log endpoint.
provenance_guard/routes/dashboard.py GET /dashboard HTML review page.
provenance_guard/routes/certificates.py Certificate request, approval, and lookup endpoints.
provenance_guard/signals/groq_signal.py Groq LLM classifier.
provenance_guard/signals/stylometry.py Stylometric heuristic scorer.
provenance_guard/signals/lexical_patterns.py Generic wording, stock transition, and template-language detector.
provenance_guard/signals/fallback_signal.py Local development fallback signal.
provenance_guard/templates/dashboard.html Dashboard template.
tests/test_routes.py Endpoint, validation, appeal, audit, dashboard, and rate-limit tests.
tests/test_scoring.py Threshold, confidence, and label tests.
tests/test_signals.py Signal-specific tests.

API

Endpoint Accepts Returns
POST /submit creator_id, content, optional title, optional content_type, optional metadata submission_id, content type, result, confidence, AI probability, label text, signals, ensemble details, optional certificate badge, status
POST /appeal / POST /appeals content_id or submission_id, creator_reasoning or reason Appeal record and original decision summary
GET /log Optional limit query parameter Structured decision and appeal audit events
GET /dashboard No body HTML analytics page with detection counts, appeal rate, average confidence, recent submissions, and appeals
POST /certificates/request creator_id, verification_method, optional evidence Verification request with pending_review status
POST /certificates/{certificate_id}/approve Reviewer/admin notes Active certificate record
GET /certificates/{creator_id} No body Active certificate status or no active certificate

POST /submit

Accepts a creator ID, content, optional title, optional content_type, and optional metadata. content_type defaults to text; the stretch multi-modal path also supports image_metadata. Returns the attribution result, confidence score, AI probability, transparency label, individual signal outputs, ensemble weights, optional certificate context, and content status.

curl -X POST http://127.0.0.1:5000/submit \
  -H "Content-Type: application/json" \
  -d '{
    "creator_id": "creator-123",
    "title": "Kitchen Window Draft",
    "content": "I wrote the first draft beside the kitchen window while rain tapped the glass..."
  }'

Example response:

{
  "submission_id": "7bbf4bb4-cb23-4492-81f7-753c17e7f718",
  "creator_id": "creator-123",
  "title": "Kitchen Window Draft",
  "content_type": "text",
  "result": "uncertain",
  "confidence_score": 0.61,
  "ai_probability": 0.61,
  "transparency_label": "Provenance Guard: Authorship is uncertain. The detection signals were mixed or not strong enough, with 61% confidence. This should not be treated as an AI-generated label.",
  "ensemble": {
    "weights": {
      "groq_llm": 0.5,
      "stylometry": 0.3,
      "lexical_patterns": 0.2
    }
  },
  "signals": {
    "groq_llm": {
      "ai_probability": 0.58,
      "confidence": 0.58,
      "details": {
        "source": "groq",
        "model": "llama-3.3-70b-versatile",
        "rationale": "Mixed evidence."
      }
    },
    "stylometry": {
      "ai_probability": 0.66,
      "confidence": 0.72,
      "details": {
        "source": "stylometric_heuristics"
      }
    },
    "lexical_patterns": {
      "ai_probability": 0.62,
      "confidence": 0.59,
      "details": {
        "source": "lexical_patterns",
        "matched_patterns": ["balanced perspective"]
      }
    }
  },
  "creator_certificate": null,
  "verified_human_badge": null,
  "status": "classified"
}

Image metadata submission example:

curl -X POST http://127.0.0.1:5000/submit \
  -H "Content-Type: application/json" \
  -d '{
    "creator_id": "artist-88",
    "title": "Gallery Upload Caption",
    "content_type": "image_metadata",
    "content": "A hand-drawn ink illustration of a city street at dusk.",
    "metadata": {
      "declared_creation_method": "hand drawn, scanned, color corrected",
      "alt_text": "Ink drawing of storefronts under evening light"
    }
  }'

POST /appeal and POST /appeals

Captures the creator's reasoning, logs it alongside the original decision, and updates the submission status to under_review. POST /appeals is an alias for the same handler.

curl -X POST http://127.0.0.1:5000/appeal \
  -H "Content-Type: application/json" \
  -d '{
    "content_id": "7bbf4bb4-cb23-4492-81f7-753c17e7f718",
    "creator_reasoning": "This is my original draft with edits from my notebook."
  }'

The route also accepts submission_id as an alias for content_id and reason as an alias for creator_reasoning.

GET /log

Returns structured audit events.

curl http://127.0.0.1:5000/log

GET /dashboard

Shows an HTML analytics page with detection counts by result, appeal rate, average confidence, recent submissions, certificate badges, statuses, and appeal counts.

Certificate endpoints

Creators request manual verification through POST /certificates/request. Reviewers approve a request through POST /certificates/{certificate_id}/approve. Platforms can check a creator's active credential with GET /certificates/{creator_id}.

For demos, the approval route can use the request ID in the URL: POST /certificates/{request_id}/approve. In that case the app generates the certificate ID. If you want a specific certificate ID, put that ID in the URL and send the request_id in the JSON body.

Certificate request example:

curl -X POST http://127.0.0.1:5000/certificates/request \
  -H "Content-Type: application/json" \
  -d '{
    "creator_id": "creator-123",
    "verification_method": "portfolio_review",
    "evidence": {
      "portfolio_url": "https://example.com/creator-123",
      "process_note": "Draft history and publication records available."
    }
  }'

Certificate-aware submission response excerpt:

{
  "submission_id": "7bbf4bb4-cb23-4492-81f7-753c17e7f718",
  "creator_id": "creator-123",
  "content_type": "text",
  "result": "likely_human",
  "confidence_score": 0.88,
  "creator_certificate": {
    "certificate_id": "cert-123",
    "status": "active",
    "verification_method": "portfolio_review"
  },
  "verified_human_badge": "Verified Human: This creator completed an additional Provenance Guard verification step. Certificate ID: cert-123."
}

Detection Signals

Provenance Guard's required baseline uses two distinct signals. The ensemble stretch feature adds a third signal and uses documented weights:

  1. Groq LLM classifier: llama-3.3-70b-versatile reads the submitted text and returns an AI probability, confidence, rationale, and evidence. This captures holistic semantic and stylistic cues that are hard to express as hand-written rules.
  2. Stylometric heuristics: pure Python statistics measure sentence length uniformity, type-token ratio, repetition, and punctuation variety. This captures structural properties of the text without relying on the LLM.
  3. Lexical pattern detector: phrase and pattern matching catches generic AI-like wording, stock transitions, and repeated template language.

The stretch final AI probability is weighted as 50% Groq + 30% stylometry + 20% lexical patterns. The weights sum to 1.0. The LLM gets the largest weight because it can assess context and tone, stylometry keeps the pipeline from depending on one signal, and lexical patterns add a narrower check for reusable generated phrasing.

Signal blind spots:

Signal What it misses
Groq LLM classifier It can be overconfident, influenced by prompt wording, unavailable if Groq is down or unconfigured, and wrong about unusual human styles. It cannot prove authorship.
Stylometric heuristics It cannot understand meaning, intent, or revision history. Short text produces weak statistics, polished humans can look uniform, and AI can imitate messy human rhythm.
Lexical pattern detector It can penalize legitimate formal writing, miss newer model phrasing, and cannot detect AI text that has been heavily edited or intentionally varied.

Stretch Features

Ensemble Detection

The ensemble stretch feature incorporates three detection signals with a documented weighting approach. The implemented ensemble uses:

Signal Weight
Groq LLM classifier 0.50
Stylometric heuristics 0.30
Lexical pattern detector 0.20

The POST /submit response includes an ensemble object with these weights and the signal names used for the final score.

Provenance Certificate

A creator can earn a verified-human credential through a manual verification request and reviewer approval. The certificate appears separately from the classification label so readers see both the work-level attribution result and the creator-level verification context.

Certificate display text:

"Verified Human: This creator completed an additional Provenance Guard verification step. Certificate ID: {certificate_id}."

Analytics Dashboard

The dashboard stretch feature implements GET /dashboard with detection patterns by result, appeal rate, and average confidence. It also keeps recent submissions, statuses, appeal counts, content-type counts, and certificate badges visible for review.

Multi-Modal Support

The multi-modal stretch feature supports content_type: "image_metadata" in addition to text through the same POST /submit route. The system does not upload binary image files. Instead, content contains the image description or caption, and metadata can include structured context such as declared creation method, upload source, tool name, or alt text. Audit logs preserve the content type and metadata.

Stretch data notes:

  • submissions.content_type defaults to text and can also be image_metadata.
  • submissions.metadata_json stores optional structured metadata for multi-modal submissions.
  • certificates stores approved verified-human credentials.
  • verification_requests stores pending and reviewed certificate requests.
  • Certificate badges are separate from the three required transparency labels.

Confidence and Uncertainty

The system stores ai_probability as the combined probability that the text is AI-generated. The displayed confidence_score is:

max(ai_probability, 1 - ai_probability)

Classification thresholds:

AI probability Result
>= 0.85 likely_ai
<= 0.25 likely_human
otherwise uncertain

The AI threshold is intentionally higher than the human threshold because a false positive, labeling a creator's human work as AI-generated, is more harmful than leaving a suspicious submission uncertain. A score near 0.51 produces an uncertain label, while a score near 0.95 produces a high-confidence AI label. Tests check those threshold differences directly.

False positive walkthrough: if a human writer submits a polished poem and the signals mistakenly score it as AI-like, mixed evidence should fall into the uncertain range instead of likely_ai. The uncertain label says the work should not be treated as AI-generated. If the score still crosses the high AI threshold, the label explicitly tells the creator they may appeal. The appeal records the creator's reasoning, changes the submission status to under_review, and keeps the original decision and appeal together in the audit log for human review.

Known Limitations

Short poems, haiku, and lyrical fragments with intentional repetition may be misread as AI-like. The stylometric signal treats repeated structure and low vocabulary diversity as possible generated-text evidence, even though those features are common in poetry and songwriting.

Polished academic, grant-style, or policy prose may score higher than intended. Those genres often use formal transitions such as "overall" or "it is important to note," which can trigger the lexical pattern detector even when a human wrote the passage.

Very short captions, titles, or two-sentence posts provide too little evidence for reliable scoring. Groq or the local fallback has limited context, stylometry cannot compute stable sentence and vocabulary patterns, and lexical matching may miss the text entirely or overreact to one phrase.

Transparency Labels

The exact label text is:

Variant Result Exact text
High-confidence AI likely_ai "Provenance Guard: This work is likely AI-generated. The detection score is {confidence_percent}% confidence. The creator may appeal this label."
High-confidence human likely_human "Provenance Guard: This work is likely human-created. The detection score is {confidence_percent}% confidence."
Uncertain uncertain "Provenance Guard: Authorship is uncertain. The detection signals were mixed or not strong enough, with {confidence_percent}% confidence. This should not be treated as an AI-generated label."

Appeals Workflow

Creators can contest a decision with POST /appeal or the POST /appeals alias. An appeal requires:

  • content_id or submission_id
  • creator_reasoning or reason

The API looks up the original submission, inserts an appeal record, updates the submission status to under_review, and writes an appeal_submitted audit event that includes both the appeal reason and the original classification decision. The system does not automatically reclassify appealed work.

Rate Limiting

POST /submit is limited per IP address:

Limit Reason
10 per minute Allows normal creative workflows while blocking rapid spam or scripted probing.
100 per day Gives active creators room to submit drafts, but caps abuse from one IP.

Appeals and log reads are not rate limited in this version because the project requirement specifically targets the submission endpoint. When the minute limit is exceeded, Flask-Limiter returns HTTP 429.

Audit Log

Every decision and appeal is stored in SQLite as a structured audit event. Each decision event includes timestamp, content ID, attribution result, confidence score, AI probability, individual signal scores, ensemble weights, status, and label text. Each appeal event includes the creator's reason and the original decision. Stretch audit events will also record image metadata submissions, certificate requests, and certificate approvals.

Sample GET /log output with at least three entries:

{
  "entries": [
    {
      "event_type": "appeal_submitted",
      "appeal_id": "appeal-001",
      "content_id": "sub-002",
      "submission_id": "sub-002",
      "creator_id": "creator-77",
      "timestamp": "2026-06-28T16:24:30.123+00:00",
      "attribution": "likely_ai",
      "confidence": 0.897,
      "ai_probability": 0.897,
      "llm_score": 0.9263,
      "stylometry_score": 0.7794,
      "lexical_pattern_score": 1.0,
      "label": "Provenance Guard: This work is likely AI-generated. The detection score is 90% confidence. The creator may appeal this label.",
      "status": "under_review",
      "appeal_filed": true,
      "appeal_reasoning": "This poem was drafted by hand and revised from my journal.",
      "original_classification": {
        "event_type": "submission_classified",
        "attribution": "likely_ai",
        "confidence": 0.897,
        "status": "classified"
      }
    },
    {
      "event_type": "submission_classified",
      "content_id": "sub-002",
      "submission_id": "sub-002",
      "creator_id": "creator-77",
      "timestamp": "2026-06-28T16:22:10.456+00:00",
      "attribution": "likely_ai",
      "confidence": 0.897,
      "ai_probability": 0.897,
      "llm_score": 0.9263,
      "stylometry_score": 0.7794,
      "lexical_pattern_score": 1.0,
      "label": "Provenance Guard: This work is likely AI-generated. The detection score is 90% confidence. The creator may appeal this label.",
      "status": "classified",
      "appeal_filed": false,
      "signals": {
        "groq_llm": {"ai_probability": 0.9263},
        "stylometry": {"ai_probability": 0.7794},
        "lexical_patterns": {"ai_probability": 1.0}
      }
    },
    {
      "event_type": "submission_classified",
      "content_id": "sub-001",
      "submission_id": "sub-001",
      "creator_id": "creator-14",
      "timestamp": "2026-06-28T16:19:05.789+00:00",
      "attribution": "likely_human",
      "confidence": 0.9336,
      "ai_probability": 0.0664,
      "llm_score": 0.0,
      "stylometry_score": 0.2212,
      "lexical_pattern_score": 0.0,
      "label": "Provenance Guard: This work is likely human-created. The detection score is 93% confidence.",
      "status": "classified",
      "appeal_filed": false
    }
  ]
}

Spec Reflection

The implementation diverged from the original required-feature plan in one important way: the required baseline was planned as two signals, Groq/local fallback plus stylometry, but the current source now runs a three-signal ensemble. The lexical pattern stretch signal was added before the other stretch features because it fit cleanly into the existing scoring pipeline and helped show individual signal scores alongside the combined result.

The implementation also changed the timing of the stretch work. Certificates, the analytics dashboard, and multi-modal support were documented before they were wired, then implemented after the Milestone 5 production layer was stable. The final stretch implementations stay lightweight: certificate verification is a manual approval flow, the dashboard is a SQLite-backed review page, and multi-modal support processes image descriptions and metadata rather than binary uploads.

AI Usage

  1. Refactor planning: I directed AI assistance to split the early single-file Flask app into a package structure with an app factory, route modules, signal modules, scoring, labels, audit, and database helpers. I revised the output to keep app.py as a minimal entrypoint and delayed the stretch-only routes until the required production layer was working.
  2. Detection and scoring implementation: I directed AI assistance to use planning.md to generate the stylometric signal, lexical pattern signal, and combined scoring logic. I checked and revised the generated behavior so the thresholds stayed at >= 0.85 for likely_ai, <= 0.25 for likely_human, and the ensemble weights stayed at 50% Groq, 30% stylometry, and 20% lexical patterns.
  3. Production layer: I directed AI assistance to add the final label behavior, appeals workflow, rate limiting, and audit evidence. I revised the endpoint contract to make POST /appeal the primary route, kept POST /appeals as a compatibility alias, and kept the production logic out of app.py so the component ownership matched the planning document.

Stretch Acceptance Criteria

  • Ensemble responses include at least three signals and documented weights.
  • Ensemble weights sum to 1.0.
  • Text submissions remain backward compatible when content_type is omitted.
  • Image metadata submissions are accepted and audited with content_type: "image_metadata".
  • Approved certificates appear in submission responses and dashboard rows.
  • The dashboard displays detection counts, appeal rate, and average confidence.
  • Appeals and certificate events are included in the audit log.

Stretch Assumptions

  • All four stretch features are implemented.
  • Multi-modal support means image descriptions or structured metadata, not binary image upload.
  • Certificate verification uses manual reviewer approval.
  • The verified-human badge is separate from the three required transparency label variants.

About

A backend system that any creative sharing platform could plug into to classify submitted content, score confidence in that classification, surface a transparency label to users, and handle appeals from creators who believe they've been misclassified.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages

0