This is the Linux app named CC-Net whose latest release can be downloaded as cc_net1.0.0sourcecode.tar.gz. It can be run online in the free hosting provider OnWorks for workstations.
Download and run online this app named CC-Net with OnWorks for free.
Ikuti petunjuk ini untuk menjalankan aplikasi ini:
- 1. Download aplikasi ini di PC Anda.
- 2. Masuk ke file manager kami https://www.onworks.net/myfiles.php?username=XXXXX dengan username yang anda inginkan.
- 3. Upload aplikasi ini di filemanager tersebut.
- 4. Jalankan emulator online OnWorks Linux atau Windows online atau emulator online MACOS dari situs web ini.
- 5. Dari OS Linux OnWorks yang baru saja Anda mulai, buka file manager kami https://www.onworks.net/myfiles.php?username=XXXXX dengan nama pengguna yang Anda inginkan.
- 6. Download aplikasinya, install dan jalankan.
Tangkapan layar
Ad
CC-Net
DESKRIPSI
cc_net provides tools to download, segment, clean, and filter Common Crawl to build large-scale text corpora, including monolingual datasets and the multilingual CC-100 collection introduced in the associated paper. It includes pipelines to fetch snapshots, extract text, de-duplicate, identify language, and apply quality filtering based on heuristics and language models. The outputs are intended for pretraining language models and for creating standardized corpora that can be reproduced or updated with new crawls. The repository documents practical concerns like HTTP failures, snapshot differences, and stats JSONs, reflecting community use across many languages. While powerful, the repo has been archived and is read-only, so users should expect to run it as-is or fork for maintenance. Even in archived state, issues and releases pages remain useful references for implementation details and dataset lineage.
Fitur
- End-to-end Common Crawl download and extraction
- Language identification and monolingual segmentation
- Quality filtering and de-duplication pipelines
- Support for building multilingual datasets like CC-100
- Reproducible statistics and corpus metadata outputs
- Scripts and configs for snapshot-by-snapshot processing
Bahasa Pemrograman
Ular sanca
KATEGORI
This is an application that can also be fetched from https://sourceforge.net/projects/cc-net.mirror/. It has been hosted in OnWorks in order to be run online in an easiest way from one of our free Operative Systems.
