Apify SDK

Apify SDK

  • Guide
  • Examples
  • Reference
  • GitHub

›Guide

Guide

  • Motivation
  • Getting Started
  • Quick Start
  • What is an Actor
  • Environment Variables
  • Data Storage
  • Puppeteer Live View

Reference

  • Apify
  • AutoscaledPool
  • BasicCrawler
  • CheerioCrawler
  • Dataset
  • KeyValueStore
  • LiveViewServer
  • PseudoUrl
  • PuppeteerCrawler
  • PuppeteerPool
  • Request
  • RequestList
  • RequestQueue
  • SessionPool
  • Session
  • utils
  • utils.log
  • utils.puppeteer
  • utils.social

Environment Variables

The following table shows the basic environment variables used by Apify SDK:

Environment variable Description
APIFY_LOCAL_STORAGE_DIR Defines the path to a local directory where key-value stores, request lists and request queues store their data. Typically it is set to ./apify_storage. If omitted, you should define the APIFY_TOKEN environment variable instead.
APIFY_TOKEN The API token for your Apify Account. It is used to access the Apify API, e.g. to access cloud storage or to run an actor in the Apify Cloud. You can find your API token on the Account - Integrations page. If omitted, you should define the APIFY_LOCAL_STORAGE_DIR environment variable instead.
APIFY_PROXY_PASSWORD Optional password to Apify Proxy for IP address rotation. If you have have an Apify Account, you can find the password on the Proxy page in the Apify app. This feature is optional. You can use your own proxies or no proxies at all.
APIFY_HEADLESS If set to 1, web browsers launched by Apify SDK will run in the headless mode. You can still override this setting in the code, e.g. by passing the headless: true option to the Apify.launchPuppeteer() function. But having this setting in an environment variable allows you to develop the crawler locally in headful mode to simplify the debugging, and only run the crawler in headless mode once you deploy it to the Apify Cloud. By default, the browsers are launched in headful mode, i.e. with windows.
APIFY_LOG_LEVEL Specifies the minimum log level, which can be one of the following values (in order of severity): DEBUG, INFO, WARNING, SOFT_FAIL and ERROR. By default, the log level is set to INFO, which means that DEBUG messages are not printed to console.
APIFY_MEMORY_MBYTES Sets the amount of system memory in megabytes to be used by the autoscaled pool. It is used to limit the number of concurrently running tasks. By default, the max amount of memory to be used is set to one quarter of total system memory, i. e. on a system with 8192 MB of memory, the autoscaling feature will only use up to 2048 MB of memory.
← What is an ActorData Storage →
Apify SDK
Docs
GuideExamplesReference
Community
Stack OverflowTwitterFacebook
More
Apify CloudDocusaurusGitHub
Copyright © 2019 Apify Technologies s.r.o.