Skip to content

Starlark Experiment#577

Open
ahirreddy wants to merge 12 commits intodatabricks:masterfrom
ahirreddy:starlark
Open

Starlark Experiment#577
ahirreddy wants to merge 12 commits intodatabricks:masterfrom
ahirreddy:starlark

Conversation

@ahirreddy
Copy link
Copy Markdown
Collaborator

No description provided.

- Install GraalVM CE 25.0.1 via SDKMAN setup script
- Add GraalVM Polyglot and Python dependencies to build.sbt
- Add PythonExample.scala to verify Python integration
- Implement simple Scala-Python binding using Val.Builtin
- Verify dictionary argument passing and integer return value from Jsonnet
- Add PythonSupport.scala with PythonEvaluator and PythonMapper
- Integrate GraalVM Polyglot for Python module loading and data conversion
- Update PythonInteropTest to verify importpy and Python function calls from Jsonnet
…luation

- Update PythonEngine to share a single Engine and cache Source objects
- Refactor PythonContextManager to manage a single Context and module cache per evaluation
- Integrated Python context management into SjsonnetMainBase using Platform hooks
- Verified shared state across multiple importpy calls in a single evaluation
…ctural caching

- Move code to sjsonnet.starlark package and directory
- Rename all Python references to Starlark (importstarlark, StarlarkEngine, etc.)
- Implement global structural caching of context-independent Val objects
- Use ThreadLocal to link cached Val objects to the current evaluation context
- Add comprehensive README explaining the architecture and performance optimizations
- Update Platform hooks and main entry points to support the new Starlark integration
- Reimplement several stdlib-like functions in Python for comparison
- Add StarlarkBenchmark with JMH integration
- Update build.mill to include Starlark benchmark resources
- Implement robust path resolution for benchmark data
- Restructure benchmark to reuse StarlarkContextManager across iterations
- Focus on pow benchmark to minimize Polyglot boundary crossing
- Update JMH settings to 20s warmup and 15s measurement
- Results show Starlark (0.005 ms/op) is 5x faster than Jsonnet (0.025 ms/op) for contained logic
- Restore makeArray, floor, ceil, sqrt, filter, and map benchmarks
- Update benchmarks.py with all necessary functions
- Keep optimized JMH settings (20s warmup, 15s measurement)
- Support selective execution via JMH regex (e.g. .pow)
- Finalize ProxyExecutable bridging in StarlarkSupport
- Consolidate StarlarkBenchmark suite with optimized timing
- Remove experimental host-bridging code
- Implement recursive freezing of user-defined globals post-module load
- Optimize valToPy to pass raw Graal Values for Starlark-backed Jsonnet values
- Add StarlarkSemanticsTest to verify local mutability vs global immutability
- Update setup script to include Go and official Starlark interpreter for validation
- Add expensive.py with a 1M iteration loop to test re-execution cost
- Add StarlarkPartialEvalBenchmark JMH test
- Verified that JIT compilation is shared while execution remains per-context
…ffle support

- Re-enable Starlark global freezing logic for hermeticity
- Implement Java-centric iteration (forEach) for improved polyglot performance
- Add Truffle API dependency and 'Tiny' language implementation for performance baselining
- Add StarlarkAllocationBenchmark to compare immutable vs mutable update strategies
- Update build.mill and build.sbt with new dependencies and resource paths
@He-Pin
Copy link
Copy Markdown
Contributor

He-Pin commented Jan 7, 2026

Seems cool

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants