Tag: rag

RAGtime trio

Posted on Posted in bead game, big bits, Valley HiTagged , , , ,

on a RAG .. or, three variations for trombone. Local language model is tempting — can AI be useful in a local arrangement. How well will iT do with photographic craft, chemistry, engineering documents useful to answering questions about recent and historic knowledge.

RAGtime fantasy: AI to revolutionize medicine. Reading medical imagery.

EX1: [A small, old, specialized network using an outdated architecture recognizes skin lesions far better than two frontier LLMs.  — https://lynchlab.net/llm-skin-cancer [ I am a dermatologist. I tested the ability of GPT-6 Astra and Claude Fable 5.1 to diagnosis skin cancer and found that they were less accurate than I expected. I compared to a 10-year old CNN and to my surprise found that this was superior. ]

1A: “– Artificial intelligence tools have been shown to help doctors detect pre-cancerous growths in the colon—but don’t even think about taking those tools away once you’ve introduced them. A new study published this week in The Lancet found that doctors who are given AI tools to assist with identifying potential cancer risks in patients get worse at making those same observations when they go back to doing it without AI’s help.” Use it, then lose your early experience. You become the intern again.

He who competes with a machine becomes a machine: Vonnegut

EX2: Failures are common enough they are a subject of research. RAG (Retrieval-Augmented Generation) systems generally fail across two main areas: Retrieval Failures(fetching or missing the right data) and Generation Failures (how the Large Language Model processes and writes the response) https://arxiv.org/abs/2401.05856

A local version of our hypothetical photographer’s AIengine:

[User Query] ──> (1. Query Analyzer) 
                       │
                       ▼
                 (2. Context Retriever) ──> [Vector DB + Reranker]
                       │
                       ▼
                 (3. Conflict Resolver) <───┐ (Self-Correction Loop)
                       │                    │
        [Clear?] ──────┴────── [Conflicting?]
           │                           │
           ▼                           ▼
    (4. Response Gen)          (Self-Correction Prompt)
           │
           ▼
     [Final Output]

Recommended 2026 RAG Stack

Layer Component Why It Wins For Legacy Tech Data
1. Parsing & Ingestion Docling or RAGFlow Traditional text splitters fail on technical PDFs. Docling natively understands complex tables, multi-column schematics, and layouts without breaking them apart.
2. Vector Database Qdrant or pgvector (PostgreSQL) For a “small” dataset (even 60 years of text is likely under a few hundred thousand chunks), pgvector keeps your stack dead simple. If your documents contain extensive part numbers or exact codes, Qdrant excels at sparse-dense hybrid search.
3. Embedding Model Cohere Embed v3 or OpenAI text-embedding-3-large You need an embedding model that handles long contexts and excels at matching technical concepts even if the phrasing/terminology shifted between 1966 and 2026.
4. Retrieval & Reranking Cohere Rerank v3 Non-negotiable for technical data. Dense vector search struggles with alphanumeric codes (e.g., “Model-TX400” vs “Model-TX401”). Pair a traditional BM25 keyword search with vector search, then pass the top results through Cohere’s cross-encoder reranker to surface the exact right chunk.
5. Orchestration LangGraph or Haystack Skip basic “naive” RAG. Technical queries require multi-step reasoning (e.g., checking a schema, then a parts list). LangGraph provides the stateful, iterative loop control required for agentic verification.
6. Large Language Model Anthropic Claude 3.5 Sonnet or OpenAI GPT-4o You need models with superior reasoning capacities to read long tables, write precise math/code, and follow strict formatting rules (like refusing to answer if the data isn’t present).
7. Evaluation & Observability RAGAS + LangSmith Critical for measuring faithfulness (did the LLM hallucinate a spec?) and answer relevance.

The perfected engine will fail if the information stored is a result of inexperienced, ie, hobby groups interacting to gain control over others.Such is the case of most online forums.

AI can’t tell experience from fantasy. A barber shop or a nail salon. Maybe a vetting parlor.

“–‘ Poets are the best authors,” the editors assert in a 1979 special issue, The Poets’ Encyclopedia, which bills itself as “the world’s basic knowledge transformed by 225 poets, artists, musicians & novelists.” “And here their business is to examine, not the individual, but the species, to remark general properties and large appearances. Audacious! The Poets imagine all knowledge.” Poets are classifying figures, in other words; they try to grasp the whole, but those in the Encyclopedia imagine the world from oblique and surprising angles. And poets are not just poets—it’s a capacious term.”

The apocalypse came due today, 11,13,26... - https://www.science.org/doi/epdf/10.1126/science.132.3436.1291
from the overnight:

most craft forums restrict topics... they erect 'sub for this/thats' built on easy storage classifier. more like teen year education. 1-12. bell ringers, hall monitors. craft shows. class clowns, bullies, promstars.
without increased shared lexicon arising from images, histories, the universals cannot grow. no response without that other reactor of 5FY53S.

metamorphosed into musty from must .. mushy
...
acrononymous

HITL humin in the loop. Too close, but used.
the little lexicon.
RAGtime.

signals -- 10s, sign, morse, flag, --

Niépce didn't call his process photographie but héliographie (φωτός/ἥλιος)
the first photography (1827 - 2027) -- plate in Gernsheim collection, Texas.

Here, let me complete that cliche for you:
no it's own leg of 
know it's own leg off
gnaw its own leg off.

he stopped his heart... his own heart. once need, clarity. extracting pronoun by enlarging. the bold print. the H1 or B tag. What was it ... the impression you wanted to transfer. [to gift]