Issues

• ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found

  VKCOM/YouTokenToMe#113 · Filter by author timotheecour opened on Dec 5, 2023

• Visual studio c++ 14.0 error while installing.

  VKCOM/YouTokenToMe#112 · Filter by author manjunath7472 opened on Oct 12, 2023

• Special tokens accessibility

  VKCOM/YouTokenToMe#111 · Filter by author xdevfaheem opened on Sep 11, 2023

• Youtokentome cannot be installed on Windows if VisualStudio not installed

  VKCOM/YouTokenToMe#96 · Filter by author yilmazay74 opened on Jul 26, 2022
  5

• Can't pip install on a new env (without Cython)

  VKCOM/YouTokenToMe#94 · Filter by author zachmoshe opened on May 19, 2022
  4

• compatibility of youtokentome.BPE outputs with the official implementation

  VKCOM/YouTokenToMe#93 · Filter by author l-k-11235 opened on Jan 11, 2022

• Installing error C1083

  VKCOM/YouTokenToMe#91 · Filter by author YFrite opened on Sep 20, 2021
  2

• error: Microsoft Visual C++ 14.0 or greater is required.

  VKCOM/YouTokenToMe#89 · Filter by author fourat-bs opened on Jul 23, 2021
  1

• Decode() got an unexpected keyword argument 'ignore_ids'

  VKCOM/YouTokenToMe#88 · Filter by author ghost opened on Jul 8, 2021

• No word tokenizer under the hood?

  VKCOM/YouTokenToMe#87 · Filter by author slowwavesleep opened on May 17, 2021

• Error during installation

  VKCOM/YouTokenToMe#86 · Filter by author ruruu127 opened on May 17, 2021

• How does YouTokenToMe's speed compare to subword-nmt?

  VKCOM/YouTokenToMe#85 · Filter by author gowtham1997 opened on Apr 16, 2021

• Using YouTokenToMe with pre-defined vocab and embeddings

  VKCOM/YouTokenToMe#84 · Filter by author alexbalandi opened on Feb 16, 2021
  2

• Tokenizing large corpus

  VKCOM/YouTokenToMe#80 · Filter by author quetz opened on Nov 1, 2020
  2

• Is it possible to unset random seed for BPE-dropout?

  enhancement
  VKCOM/YouTokenToMe#75 · Filter by author skurzhanskyi opened on Sep 17, 2020
  2

• Controlling word tokenization

  VKCOM/YouTokenToMe#73 · Filter by author MexicanMan opened on Aug 20, 2020

• How to train with multiple corpus files?

  VKCOM/YouTokenToMe#72 · Filter by author hccho2 opened on Aug 15, 2020

• Process killed?

  VKCOM/YouTokenToMe#71 · Filter by author miguelvictor opened on Jul 31, 2020
  4

• how to get vocab

  VKCOM/YouTokenToMe#69 · Filter by author wqfengnlpr opened on Apr 30, 2020
  2

• Vocabulary contains underscore multiple times?

  VKCOM/YouTokenToMe#68 · Filter by author RuABraun opened on Apr 20, 2020

• "▁" character can be separated when using BPE-dropout

  VKCOM/YouTokenToMe#67 · Filter by author TIXFeniks opened on Apr 4, 2020
  11

• How to generate vocab.json and merges.txt for YTTM tokenizer?

  VKCOM/YouTokenToMe#66 · Filter by author nikhilno1 opened on Mar 8, 2020
  1

• Special Tokens

  VKCOM/YouTokenToMe#65 · Filter by author martingajek opened on Mar 6, 2020
  1

• Adding new terms into pre-trained model vocab | Issue in tokenizing OOV keywords

  VKCOM/YouTokenToMe#63 · Filter by author spate141 opened on Feb 24, 2020

• Doesn't consider combining characters.

  VKCOM/YouTokenToMe#59 · Filter by author IDDT opened on Jan 21, 2020