This is the Linux app named Punica whose latest release can be downloaded as punica-1.1.0.tar.gz. It can be run online in the free hosting provider OnWorks for workstations.
Download and run online this app named Punica with OnWorks for free.
Follow these instructions in order to run this app:
- 1. Downloaded this application in your PC.
- 2. Enter in our file manager https://www.onworks.net/myfiles.php?username=XXXXX with the username that you want.
- 3. Upload this application in such filemanager.
- 4. Start the OnWorks Linux online or Windows online emulator or MACOS online emulator from this website.
- 5. From the OnWorks Linux OS you have just started, goto our file manager https://www.onworks.net/myfiles.php?username=XXXXX with the username that you want.
- 6. Download the application, install it and run it.
SCREENSHOTS:
Punica
DESCRIPTION:
Punica is a system designed to efficiently serve multiple LoRA-fine-tuned large language models within a shared GPU environment. LoRA is a parameter-efficient fine-tuning method that allows developers to adapt large pretrained models to specific tasks by adding lightweight adapter layers rather than retraining the entire model. Punica introduces a serving architecture that allows multiple LoRA adapters to share the same base model during inference, significantly reducing memory consumption and computational overhead. The system includes specialized CUDA kernels that enable batched GPU operations across different LoRA models simultaneously. This design allows a single GPU cluster to host many task-specific models while maintaining high throughput and minimal latency. The architecture also includes scheduling mechanisms that coordinate requests from multiple tenants and distribute workloads efficiently across available resources.
Features
- Multi-tenant serving system for LoRA-adapted language models
- Shared base model architecture reducing memory usage
- Custom CUDA kernels enabling efficient batched LoRA inference
- GPU scheduling system for high throughput request handling
- Support for hosting multiple task-specific adapters simultaneously
- Optimized architecture for scalable LLM inference services
Programming Language
Python
Categories
This is an application that can also be fetched from https://sourceforge.net/projects/punica.mirror/. It has been hosted in OnWorks in order to be run online in an easiest way from one of our free Operative Systems.