Return to Answer

Your todo function should probably be std::unimplemented. You could also make it an uncaught exception, such as std::runtime_error. The best way to handle an out-of-memory error is to throw std::bad_alloc as the Standard Library does, or to allocate a smart object such as std::unique_ptr or std::vector that manages its own memory.

Your todo function should probably be an uncaught exception, such as std::logic_error. The best way to handle an out-of-memory error is to throw std::bad_alloc as the Standard Library does, or to allocate a smart object such as std::unique_ptr or std::vector that manages its own memory.

Tokens are tiny snippets of the source file, so if you memory-map the file (which requires Boost or some other library to do portably), your tokens could each be a string_view of the file contents in memory, eliminating any need to copy their data at all. In that case, the token-list class should own the memory mapping, and its destructor should unmap close the file at the same time it destroys the vector of tokens, to prevent dangling references.

Tokens are tiny snippets of the source file, so if you memory-map the file (which requires Boost or some other library to do portably), your tokens could each be a string_view of the file contents in memory, eliminating any need to copy their data at all. In that case, the token-list class should own the memory mapping, and its destructor should unmap/close the file at the same time it destroys the vector of tokens, to prevent dangling references.

Tokens are tiny snippets of the source file, so if you memory-map the file (which requires Boost or some other library), your tokens could each be a string_view of the file contents in memory, eliminating any need to copy their data at all. In that case, the token-list class should own the memory mapping, and its destructor should unmap close the file at the same time it destroys the vector of tokens, to prevent dangling references.

Tokens are tiny snippets of the source file, so if you memory-map the file (which requires Boost or some other library to do portably), your tokens could each be a string_view of the file contents in memory, eliminating any need to copy their data at all. In that case, the token-list class should own the memory mapping, and its destructor should unmap close the file at the same time it destroys the vector of tokens, to prevent dangling references.