NCRF++, a Neural Sequence Labeling Toolkit. Easy use to any sequence labeling tasks (e.g. NER, POS, Segmentation). It includes character LSTM/CNN, word LSTM/CNN and softmax/CRF components.

Fully neural approach for text chunking

A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

A package for parsing PDFs and analyzing their content using LLMs.

The RAG Experiment Accelerator is a versatile tool designed to expedite and facilitate the process of conducting experiments and evaluations using Azure Cognitive Search and RAG pattern.

A TensorFlow implementation of Neural Sequence Labeling model, which is able to tackle sequence labeling tasks such as POS Tagging, Chunking, NER, Punctuation Restoration and etc.

A new chunking strategy developed by ZeroEntropy for general semantic chunking using Llama-70B.

An LLM GUI application; enables you to interact with your files, offering dynamic parameters that can modify response behavior during runtime.

a modular multimodal framework for ai applications

smart-llm-loader is a lightweight yet powerful Python package that transforms any document into LLM-ready chunks. Spend less time on preprocessing headaches and more time building what matters. From RAG systems to chatbots to document Q&A, SmartLLMLoader handles the heavy lifting so you can focus on creating exceptional AI applications.

FastCDC implementation in Python https://pypi.org/project/fastcdc/

Extract and align grammar patterns from English sentences.

Build document-native LLM applications

Retrieval-augmented generation (RAG) for remote & local LLM use

LLM Chatbot w/ Retrieval Augmented Generation using Llamaindex. It demonstrates how to impl. chunking, indexing, and source citation.

BetterHTMLChunking is a Python library for intelligent HTML segmentation. It builds a DOM tree from raw HTML and extracts content-rich regions of interest, making content analysis effortless. Great for LLM based processing.

Linux GUI for AV1 Encoders

A Python-based parallel file chunking system designed for processing large codebases into LLM-friendly chunks.

Sequence Labeling in Tensorflow

Mishtar: Named and temporal entities chunker