This is the Linux app named Smallpond whose latest release can be downloaded as smallpondv0.15.0sourcecode.tar.gz. It can be run online in the free hosting provider OnWorks for workstations.
Download and run online this app named Smallpond with OnWorks for free.
Follow these instructions in order to run this app:
- 1. Downloaded this application in your PC.
- 2. Enter in our file manager https://www.onworks.net/myfiles.php?username=XXXXX with the username that you want.
- 3. Upload this application in such filemanager.
- 4. Start the OnWorks Linux online or Windows online emulator or MACOS online emulator from this website.
- 5. From the OnWorks Linux OS you have just started, goto our file manager https://www.onworks.net/myfiles.php?username=XXXXX with the username that you want.
- 6. Download the application, install it and run it.
SCREENSHOTS
Ad
Smallpond
DESCRIPTION
smallpond is a lightweight distributed data processing framework built by DeepSeek, designed to scale DuckDB workloads over clusters using their 3FS (Fire-Flyer File System) backend. The idea is to preserve DuckDB’s fast analytics engine but lift it from single-node to multi-node settings, giving you the ability to operate on large datasets (e.g. petabyte scale) without moving to a heavyweight system like Spark. Users write Python-like code (via DataFrame APIs or SQL strings) to express their transformations; behind the scenes, tasks are scheduled (often via Ray) and pushed into DuckDB instances operating on partitioned data. Because the storage layer (3FS) is optimized for random access and high throughput, smallpond can shuffle data, repartition, and manage intermediate results across nodes.
Features
- Distributed extension of DuckDB: support for running SQL / DataFrame operations across nodes
- Uses 3FS as the shared data backend to manage data storage and shuffle operations
- APIs for transformations via SQL strings or Python functions (map, partial_sql)
- Support for repartitioning by number of partitions, row count, or hash on a column
- Two execution modes: high-level dynamic (Ray-based) and low-level static graph execution
- Optimized for large-scale workloads (benchmarked at ~100 TiB sorting)
Programming Language
Python
Categories
This is an application that can also be fetched from https://sourceforge.net/projects/smallpond.mirror/. It has been hosted in OnWorks in order to be run online in an easiest way from one of our free Operative Systems.
