tokenization

Multi Cloud Data Tokenization Solution By Using Dataflow and Cloud DLP

This is a java version of Chinese tokenization descried in BERT.

Identify and tokenize sensitive data automatically using Cloud DLP and Dataflow

Public code samples and resources for the Thales CipherTrust Application Protection products of the CipherTrust Data Security Platform

serverless ☁️ 🚀 , pseudonymizing proxy between Worklytics and your workplace 💼 SaaS data sources' APIs. Data Loss Prevention (DLP) 🛡️🔒 and compliance layer deployable to AWS Lambda or GCP Cloud Functions.

Babel Street Analytics Client Library for Java

🔧 My studies on context-free grammar, using ANTLR4 (C++) to generate the parser files. Some basics are developed, such as token processing, recursion, variable definition, array processing, Abstract Syntax Tree (AST) manipulation, UNICODE support, and error handling.

Java library to optimize payload format for minimum token consumption

The tokenisation of spoken text. Received by the Watson STT and sent to the Apache OpenNLP. Additional code creates individual tokens, depending on the recorded sentences

Multilingual Natural Language Processing for Java

Resolving conflict merges with ASTs

IXOPAY SDK for card tokenization on Android.

Protegrity AI Developer Edition – Java SDK for Data Discovery and Protection APIs

Blockchain-based P2P marketplace for trading solar energy and renewable certificates. Features smart-contract auctions, tokenized kWh/RECs, real-time pricing, and NREL API integration for authentic energy data.

Android Java "add payment card" form - This app demonstrates how simple it is to add payment card data to your app with VeryGoodSecurity

Java Springboot backend application for onboarding new customers (REST Apis) - Monolithic Application

Compiler Construction contains three phase 1)Lexical Analyzer 2)Syntax Analyzer 3)Semantic Analyzer

Language processing interface: some tools to process different natural languages

Java-based lexical analyzer for the educational IU language. Tokenizes source with hand-built DFAs (identifiers, numbers, strings, chars, booleans, operators, comments), tracks line numbers, reports unknown tokens, and builds a simple symbol table. Includes sample input and build script.

Program that preprocesses a collection of documents to calculate the frequency of the most common terms and identify the keywords of each document. The first time will do it without using the stemming technique and without removing the stopwords. The second time will use these techniques.