Probity People

By Alex Turner, March 10, 2026

Probity People

Introduction to Health Checks in App Engine Flexible Environment

In modern application development, ensuring that deployed applications maintain high availability is paramount. Google Cloud Platform’s (GCP) App Engine Flexible Environment offers a powerful approach to this by utilizing health checks, specifically liveness and readiness checks. These checks play a crucial role in determining how an application responds to traffic based on its current health status. Configuring these checks properly can significantly influence an application’s reliability during deployments and operational failures.

Understanding Liveness Checks and Readiness Checks

Health checks are integrated mechanisms that App Engine uses to monitor instance health. They consist of two primary types:

  • Liveness Check: This check determines if the application instance is running and responsive. If the liveness check fails, the App Engine automatically restarts the instance to recover from possible failures.
  • Readiness Check: This evaluates whether the instance is ready to accept user traffic. If the readiness check fails, the App Engine will stop routing traffic to that instance while keeping it alive to recover.

These checks are not interchangeable; understanding their distinct purposes is critical for the smooth operation of applications. For example, an instance may be alive yet unable to serve traffic due to pending database connections or data loading, making it vital for each check to serve its designated purpose without overlap.

Configuring Health Checks in Your Application

To implement health checks in your application, they need to be defined in the app.yaml file, guiding App Engine on how to perform liveness and readiness checks. Below is a sample configuration for both checks:

# app.yaml - Health check configuration for App Engine Flexruntime: customenv: flexliveness_check: path: "/_ah/live" check_interval_sec: 30 # How often to check (seconds) timeout_sec: 4 # Max time to wait for response failure_threshold: 4 # Failures before marking unhealthy success_threshold: 2 # Successes before marking healthy again initial_delay_sec: 300 # Wait before starting checks after instance bootreadiness_check: path: "/_ah/ready" check_interval_sec: 5 # Check more frequently than liveness timeout_sec: 4 # Max time to wait for response failure_threshold: 2 # Fewer failures needed to stop traffic success_threshold: 2 # Successes before routing traffic app_start_timeout_sec: 300 # Max time to wait for first successful checkresources: cpu: 1 memory_gb: 1 disk_size_gb: 10

Liveness Check Settings Explained

  • path: The endpoint the App Engine hits to verify liveness; it must return a 200 response for a healthy status.
  • check_interval_sec: Frequency of the liveness checks. A lower value decreases detection time for errors but incurs additional overhead.
  • timeout_sec: The duration to wait for a response before considering the check failed.
  • failure_threshold: The number of consecutive failed checks before the instance is marked unhealthy. It’s often set higher for applications that sometimes respond slowly.
  • success_threshold: The needed consecutive successes to restore the instance to a healthy state after a failure.
  • initial_delay_sec: Time taken before starting the checks after instance startup, allowing the application time for initialization.

Readiness Check Settings Explained

  • path: The endpoint for the readiness check, which functions separately from the liveness check.
  • check_interval_sec: Run this check more frequently than the liveness check, typically every five seconds.
  • failure_threshold: The threshold is set lower compared to the liveness check, allowing faster traffic redirection and improving user experience.
  • success_threshold: Similar to the liveness check, this represents the amount of successful responses required to deem the instance ready for traffic.
  • app_start_timeout_sec: Maximum allowable time for health checks during a new deployment; if this limit is exceeded, the deployment fails and is rolled back.

Implementing Effective Health Check Endpoints

Basic Health Check Implementation

When starting out, implementing simple endpoints may suffice:

# Simple health checks - just verify the process is running@app.route("/_ah/live")def liveness(): return "OK", 200@app.route("/_ah/ready")def readiness(): return "OK", 200

However, merely returning a “200 OK” status misses the whole point. Should your application be functional yet cannot connect to the database, you are still exposing users to errors. Thus, the implementation of more nuanced health checks that verify dependencies is crucial.

Comprehensive Health Check Example

Here’s a more thorough implementation that involves checking dependencies:

# health.py - Comprehensive health check endpointsimport timeimport redisimport sqlalchemyfrom flask import Flask, jsonifyapp = Flask(__name__)# Track application statestartup_time = time.time()is_initialized = Falsedb_engine = Noneredis_client = Nonedef initialize_app(): """Run during startup - called from warmup or first request.""" global is_initialized, db_engine, redis_client try: db_engine = create_database_engine() redis_client = create_redis_client() is_initialized = True except Exception as e: app.logger.error(f"Initialization failed: {e}") is_initialized = False@app.route("/_ah/live")def liveness(): """Liveness check - is the process functioning?""" try: _ = {"status": "alive", "uptime": time.time() - startup_time} return "OK", 200 except Exception: return "FAIL", 503@app.route("/_ah/ready")def readiness(): """Readiness check - can this instance handle real traffic?""" checks = {} # Check if initialization has completed if not is_initialized: return jsonify({"ready": False, "reason": "not initialized"}), 503 # Check database connectivity try: with db_engine.connect() as con: con.execute(sqlalchemy.text("SELECT 1")) checks["database"] = "ok" except Exception as e: checks["database"] = f"failed: {str(e)}" return jsonify({"ready": False, "checks": checks}), 503 # Check Redis connectivity try: redis_client.ping() checks["redis"] = "ok" except Exception as e: checks["redis"] = f"failed: {str(e)}" return jsonify({"ready": False, "checks": checks}), 503 return jsonify({"ready": True, "checks": checks}), 200

Understanding the Key Principle: Lightweight Liveness

One common mistake developers make is including dependency checks within the liveness endpoint. This is misguided; if a database failure triggers the liveness check to fail, App Engine will restart the instance. This action does not resolve the underlying database issue and leads to a continuous restart loop, creating further complications. Therefore, liveness checks should focus solely on the state of the application process itself.

Tuning Deployment Speeds through Readiness Checks

For effective deployment strategies, the readiness checks influence how quickly new instances can begin accepting traffic. To enhance deployment speed:

readiness_check: path: "/_ah/ready" check_interval_sec: 2 # Check every 2 seconds during startup timeout_sec: 1 failure_threshold: 2 success_threshold: 1 # Accept after 1 success app_start_timeout_sec: 180 # Shorter startup deadline

Handling Graceful Shutdowns

During instance termination, GCP sends a SIGTERM signal and ceases readiness checks. Proper handling of this signal ensures that your application finishes processing in-flight requests before shutting down:

# Graceful shutdown handlingimport signalimport sysshutting_down = Falsedef handle_sigterm(signum, frame): global shutting_down shutting_down = True app.logger.info("SIGTERM received, starting graceful shutdown") # Finish in-flight requests and close connections sys.exit(0)signal.signal(signal.SIGTERM, handle_sigterm)@app.route("/_ah/ready")def readiness(): if shutting_down: return "Shutting down", 503 # ... normal readiness checks

Debugging Health Check Failures

When health checks fail, reviewing the logs is vital for diagnosing issues:

# View health check related logsgcloud logging read 'resource.type="gae_app" AND "health"' \ --project=your-project-id \ --limit=50# View instance lifecycle eventsgcloud logging read 'resource.type="gae_app" AND "instance"' \ --project=your-project-id \ --limit=50

If your instances repeatedly restart, it indicates a failing liveness check. If they run but don’t accept traffic, the readiness check has likely failed. The logs will provide the necessary insights to identify the faulty check.

Conclusion

In summary, the accurate configuration of liveness and readiness checks is pivotal for the seamless operation of your applications on App Engine Flex. Liveness checks should be lightweight, confirming only that the application process is functioning. Readiness checks must verify all critical dependencies, ensuring that traffic is only routed to healthy instances. Setting appropriate thresholds helps maintain application stability and responsiveness, contributing to a reliable user experience. By adhering to these guidelines, developers can optimize their applications for performance and resilience.

For detailed insights and resources regarding these configurations, please refer to Probity People.

Disclaimer: The information provided in this article is intended for educational purposes and should not be considered as professional or expert advice. Always consult with a qualified professional before making decisions based on the content provided.