EnglishFrenchSpanish

Ad


OnWorks favicon

Scrapy-Redis download for Windows

Free download Scrapy-Redis Windows app to run online win Wine in Ubuntu online, Fedora online or Debian online

This is the Windows app named Scrapy-Redis whose latest release can be downloaded as Version0.7.1Released.zip. It can be run online in the free hosting provider OnWorks for workstations.

Download and run online this app named Scrapy-Redis with OnWorks for free.

Follow these instructions in order to run this app:

- 1. Downloaded this application in your PC.

- 2. Enter in our file manager https://www.onworks.net/myfiles.php?username=XXXXX with the username that you want.

- 3. Upload this application in such filemanager.

- 4. Start any OS OnWorks online emulator from this website, but better Windows online emulator.

- 5. From the OnWorks Windows OS you have just started, goto our file manager https://www.onworks.net/myfiles.php?username=XXXXX with the username that you want.

- 6. Download the application and install it.

- 7. Download Wine from your Linux distributions software repositories. Once installed, you can then double-click the app to run them with Wine. You can also try PlayOnLinux, a fancy interface over Wine that will help you install popular Windows programs and games.

Wine is a way to run Windows software on Linux, but with no Windows required. Wine is an open-source Windows compatibility layer that can run Windows programs directly on any Linux desktop. Essentially, Wine is trying to re-implement enough of Windows from scratch so that it can run all those Windows applications without actually needing Windows.

SCREENSHOTS

Ad


Scrapy-Redis


DESCRIPTION

You can start multiple spider instances that share a single redis queue. Best suitable for broad multi-domain crawls. Scraped items gets pushed into a redis queued meaning that you can start as many as needed post-processing processes sharing the items queue. Scheduler + Duplication Filter, Item Pipeline, Base Spiders. Default requests serializer is pickle, but it can be changed to any module with loads and dumps functions. Note that pickle is not compatible between python versions. Version 0.3 changed the requests serialization from marshal to cPickle, therefore persisted requests using version 0.2 will not able to work on 0.3. The class scrapy_redis.spiders.RedisSpider enables a spider to read the urls from redis. The urls in the redis queue will be processed one after another, if the first request yields more requests, the spider will process those requests before fetching another url from redis.



Features

  • Distributed crawling/scraping
  • Distributed post-processing
  • Scrapy plug-and-play components
  • Python 2.7, 3.4 or 3.5 required
  • Redis >= 2.8 required
  • Scheduler + Duplication Filter, Item Pipeline, Base Spiders


Programming Language

Python



This is an application that can also be fetched from https://sourceforge.net/projects/scrapy-redis.mirror/. It has been hosted in OnWorks in order to be run online in an easiest way from one of our free Operative Systems.


Free Servers & Workstations

Download Windows & Linux apps

  • 1
    NSIS: Nullsoft Scriptable Install System
    NSIS: Nullsoft Scriptable Install System
    NSIS (Nullsoft Scriptable Install
    System) is a professional open source
    system to create Windows installers. It
    is designed to be as small and flexible
    as possi...
    Download NSIS: Nullsoft Scriptable Install System
  • 2
    authpass
    authpass
    AuthPass is an open source password
    manager with support for the popular and
    proven Keepass (kdbx 3.x AND kdbx 4.x ...
    Download authpass
  • 3
    Zabbix
    Zabbix
    Zabbix is an enterprise-class open
    source distributed monitoring solution
    designed to monitor and track
    performance and availability of network
    servers, device...
    Download Zabbix
  • 4
    KDiff3
    KDiff3
    This repository is no longer maintained
    and is kept for archival purposes. See
    https://invent.kde.org/sdk/kdiff3 for
    the newest code and
    https://download.kde.o...
    Download KDiff3
  • 5
    USBLoaderGX
    USBLoaderGX
    USBLoaderGX is a GUI for
    Waninkoko's USB Loader, based on
    libwiigui. It allows listing and
    launching Wii games, Gamecube games and
    homebrew on Wii and WiiU...
    Download USBLoaderGX
  • 6
    Firebird
    Firebird
    Firebird RDBMS offers ANSI SQL features
    & runs on Linux, Windows &
    several Unix platforms. Features
    excellent concurrency & performance
    & power...
    Download Firebird
  • More »

Linux commands

Ad