Bioinformatician and data scientist in Copenhagen. Human genetics at population scale: reference panels for African populations at the Wellcome Sanger Institute, deep learning on UK Biobank health records, and clinical machine learning on national registry and EHR data, together with the methods that make exact estimates computable at that scale.
tommycarstensen.github.io · tommycarstensen.com · Google Scholar
Exact out-of-core Cox proportional hazards regression. Each Newton–Raphson step becomes a sum over parquet row groups, so peak memory is one row group plus O(p^2) state rather than the whole table. Coefficients agree with R's survival::coxph to 2e-14 at two million rows, and with lifelines to 3e-14 on data rounded to 120 distinct event times, where ties necessarily straddle chunk boundaries. Standard errors agree to 2e-13.
Install: pip install coxstream · CRAN · R source
Streaming generalized linear models with bounded memory, implementing the renewable estimation of Luo and Song (2020). One pass per iteration accumulates the sufficient statistics, so the streaming path holds one chunk plus O(p^2) state, independent of the number of rows; the chunk-buffered path trades that for standard errors. Gaussian, binomial and Poisson, checked against statsmodels in the test suite.
Install: pip install renew-glm · doi:10.5281/zenodo.21950314
Slide deck on where memory actually goes: streaming engines against pandas, parquet against CSV, and what the wrong dtype costs. Python and R, 50 to 800 million rows on synthetic registry-shaped data, with the measurement methodology included.
AI-powered extraction of numerical data from scientific figures. Upload images, PDFs, or ZIPs; Claude Vision reads boxplots, bar charts, and line plots and returns structured tables (Excel, CSV, JSON, R). Benchmarked against ChartX and PlotQA datasets across Claude, GPT, Gemini, and DePlot baselines. Hosted app (Streamlit sign-in): plotpick.streamlit.app
polars #27860: reported with a reproducer at 200 million rows. Parquet min/max statistics exclude NaN while polars orders NaN as the largest value, so float >= filters could never skip row groups. Labelled accepted upstream; the nan_count-gated fix is verified locally and not yet merged.
Merged pull requests to samtools/bcftools and biopython.