Today we’re going to shed a light on an interesting topic. PHP internals, this term is used to refer to all those things that are not directly accessible to userspace code (php script). This post explains the concept of (ZTS) Zend thread safety in multi-threaded environment, PHP lifecycle and a detailed analysis of TSRM mechanism.

Zend Thread Safety Principles:

Thread Security means to guarantee a safe access to public resources in a multithreaded environment. Generally each thread has a private stack but all threads share a common heap. In C language, when a variable is declared outside of any function, it becomes a global variable, then this variable will be assigned to the process of shared storage space, different threads refer to the same memory address space. At first PHP had no concern for thread security because there was no resource competition or concurrency between threads ( single-threaded environment ).

After the appearance of single-process multithreaded web servers a new approach to handling global data became necessary. Eventually this would appear as a new layer in PHP called TSRM (Thread Safe Resource Management), the concrete implementation of Zend Thread-Safe mechanism. In this paper we’ll discuss examples of specific implementations in Apache HTTP web server (Worker MPM) which uses threads. Threads proceed through TSRM which manage access to its global variables. Each thread assigned an area, and is identified by a unique ID as shown below :

The following figure shows a schematic view of ZTS concept:

General Overview about PHP Request Life Cycle :

Every PHP instance go through numerous events involving Request Init, Module Init and finally Shutdown events. The initial startup, before page requesting (SCRIPT) begins, PHP interpreter calls every extension’s MINIT (Module Initialization) method to initialize extension modules.

During this phase extensions declares constants, register resource, stream, and filter handlers that all will be used in next requests. Once a request is made, PHP interpreter sets up an environment to operate requests which includes a symbol table (php variables and could be Engine’s vars too).

PHP loops through its extensions again and calls each extension’s RINIT ( Request Initialization ) method which run at the start of every page request. During this phase extensions reset global variables to default values ( either in multithreaded env), pass variables into the script’s symbol table.

Finally when the request has completed execution, due to the end of scripts file or by exiting through a die() or exit() statements, the cleanup process start by calling each extension’s RSHUTDOWN (Request Shutdown) method. RSHUTDOWN corresponds to auto_append_file configuration directive. But the difference between RSHUTDOWN and auto_append_file, is that RSHUTDOWN don’t get interrupted by a call from exit() – die() will always be executed, while auto_append_file is skipped by such calls. After RSHUTDOWN methods have completed, symbol table and non-persistent resources are destroyed.

Now it’s SAPI and web server turn to shutdown, PHP loops through each extension’s MSHUTDOWN (Module Shutdown) method and unregister any kind of remaining resources.

Illustration of php running in a multi-threaded server, handling requests life cycle as shown below:

TSRM API & Related Algorithms:

This section is about analyzing the implementation details of the main TSRM algorithms. Let’s start first with defining data structures :

Data Structures :

TSRM concerned with 2 important data structures :

1. tsrm_tls_entry .                                                                                   
2. tsrm_resource_type.
   

typedef struct _tsrm_tls_entry tsrm_tls_entry;

struct _tsrm_tls_entry {
void **storage; // global variable storage array 
int count; // resources recorded count,
THREAD_T thread_id;
tsrm_tls_entry *next;
};

Each tsrm_tls_entry node corresponds to a thread. Multiple tsrm_tls_entry linked together to form tsrm_tls_table , a static global variable defined in the memory manager table :

/* The memory manager table */
static tsrm_tls_entry **tsrm_tls_table=NULL;
static int tsrm_tls_table_size; //
static ts_rsrc_id id_count; 

tsrm_resource_type‘s internal structure is relatively simple:

typedef struct {
size_t size; // shared memory size ( resources )
ts_allocate_ctor ctor // resources constructor
ts_allocate_dtor dtor // resources destructor 
int done; // a flag to identify resource usage. 
} tsrm_resource_type; 

tsrm_resource_type resource (or global variables) as a unit, every time a new resource is allocated, it will create a tsrm_resource_type element and expand global resource table. All tsrm_resource_type is an array approach consisting tsrm_types_table, with rsrc_id as the index of resources.  This data structure is mainly used in TSRM to manage each thread creation and release resources. tsrm_types_table is a global static variable defined in the resource sizes table as resource_types_table :

/* The resource sizes table */
static tsrm_resource_type *resource_types_table=NULL;
static int resource_types_table_size;

Implementation Details :

Basically the TSRM mechanism has 3 major functions to pick for representative analysis. The first is tsrm_startup function that is called in the process sapi initial stage, to initialize TSRM environment. The following is the function code :

/* Startup TSRM (call once for the entire process) */
TSRM_API int tsrm_startup(int expected_threads, int expected_resources, int debug_level, char *debug_filename)
{
#if defined(GNUPTH)
 pth_init();
#elif defined(PTHREADS)
 pthread_key_create( &tls_key, 0 );
#elif defined(TSRM_ST)
 st_init();
 st_key_create(&tls_key, 0);
#elif defined(TSRM_WIN32)
 tls_key = TlsAlloc();
#elif defined(BETHREADS)
 tls_key = tls_allocate();
#endif

 tsrm_error_file = stderr;
 tsrm_error_set(debug_level, debug_filename);
 tsrm_tls_table_size = expected_threads;

 tsrm_tls_table = (tsrm_tls_entry **) calloc(tsrm_tls_table_size, sizeof(tsrm_tls_entry *));
 if (!tsrm_tls_table) {
 TSRM_ERROR((TSRM_ERROR_LEVEL_ERROR, "Unable to allocate TLS table"));
 return 0;
 }
 id_count=0;

 resource_types_table_size = expected_resources;
 resource_types_table = (tsrm_resource_type *) calloc(resource_types_table_size, sizeof(tsrm_resource_type));
 if (!resource_types_table) {
 TSRM_ERROR((TSRM_ERROR_LEVEL_ERROR, "Unable to allocate resource types table"));
 free(tsrm_tls_table);
 tsrm_tls_table = NULL;
 return 0;
 }

 tsmm_mutex = tsrm_mutex_alloc();
 // Initialize the thread start and end handles
 tsrm_new_thread_begin_handler = tsrm_new_thread_end_handler = NULL;

 TSRM_ERROR((TSRM_ERROR_LEVEL_CORE, "Started up TSRM, %d expected threads, %d expected resources", expected_threads, expected_resources));
 return 1;
}

The main task here is to initialize the two data structures mentioned above.

First two parameters are : expected_threads and expected_resources. These two parameters are passed by the SAPI, it represents the number of threads and the number of resources expected in the environment. Next the function initialize tsrm_tls_table_size (mentioned in the memory manager table) with expected_threads value and following allocate memory space for tsrm_tls_table (via calloc) and initialize id_count with 0 for the first use. The same procedure is applied to resource_types_table, resource_types_table_size is initialized with expected_resources value then allocate memory space for resource_types_table
(mentioned in the the resource sizes table). Finally it allocates a mutex to handle resources allocation and usuage without mutual interference between threads.
Below the implementation code for tsrm_mutex_alloc() :

/* Allocate a mutex */
TSRM_API MUTEX_T tsrm_mutex_alloc(void)
{
MUTEX_T mutexp;
#ifdef TSRM_WIN32
mutexp = malloc(sizeof(CRITICAL_SECTION));
InitializeCriticalSection(mutexp);
#elif defined(GNUPTH)
mutexp = (MUTEX_T) malloc(sizeof(*mutexp));
pth_mutex_init(mutexp);

/* code ... */

#endif
#ifdef THR_DEBUG
printf("Mutex created thread: %d\n",mythreadid());
#endif
return( mutexp );
}

The second function ts_allocate_id, significantly the most important in the TSRM mechanism, needs to be carefully analyzed. Generally speaking this function is used to allocate global resources in a multithreaded environment and return the resource ID so let’s discover how it works.

You can find the source code here :  TSRM : ts_allocate_id.                                            
                                                                 TSRM : ts_allocate_id (link 2).

We call this function during the extension’s MINIT process, TSRM layer is notified with the size of data which needs to be stored by the extension. Following that TSRM adds the number of bytes to its running total of data space requirements then returns a new, unique identifier for that segment’s portion of the global thread’s data pool. It’s used to apply a global variable in a multithreaded environment (in ex : Apache Worker MPM ) and return the ressource ID.

Analysis :

Once we call ts_allocate_id function, 4 parameters are passed by the extension. Example ( Apache SAPI extension ):

static PHP_MINIT_FUNCTION(apache)
{
#ifdef ZTS
ts_allocate_id(&php_apache_info_id, sizeof(php_apache_info_struct), (ts_allocate_ctor) php_apache_globals_ctor, NULL);
#else
php_apache_globals_ctor(&php_apache_info TSRMLS_CC);
#endif
REGISTER_INI_ENTRIES();
return SUCCESS;
}

Next PHP kernel lock the calling thread using tsrm_mutex_lock function to handle concurrency issues. Once locked id_count is incremented, to generate a new resource ID, after generating a resource ID, it gives the current location of the newly allocated storage, each resource will be stored in tsrm_types_table. In fact, we can consider tsrm_types_table a sort of HashTable, its key is the resource ID and value is tsrm_resource_type structure.

Later it will assing resource data to resource_types_table and set done to 0 which means that the resource is currently in use. After the allocation of resource ID,  ts_allocate_id function will traverse all the threads and for each thread’s tsrm_tls_entry allocate the resources required memory space based on id_count. More precisely it will check if the global variables recorded count is less than the id_count and enlarge the existing global variables (resources) storage array. Then allocate the size of ressources starting from the last count and call the resource constructor.

We should note that tsrm_tls_table (in other words tsrm thread local storage table) is a HashTable, an array of pointers to each thread private data (tsrm_tls_entry ). Index is calculated using the following macro :

// Thr stand for thread_id
// ts  stand for tsrm_tls_table_size 
#define THREAD_HASH_OF(thr, ts) (unsigned long)thr%(unsigned long)ts
// Usage Example
hash_value = THREAD_HASH_OF(thread_id, tsrm_tls_table_size);
thread_resources = tsrm_tls_table[hash_value];

Resource Allocation Process Flow chart :

Finally it’s time for tsrm_shutdown function to shutdown TSRM and do some memory cleanup job.

/* Shutdown TSRM (call once for the entire process) */
TSRM_API void tsrm_shutdown(void)
{
    int i;

    if (tsrm_tls_table) {
        for (i=0; i<tsrm_tls_table_size; i++) {
            tsrm_tls_entry *p = tsrm_tls_table[i], *next_p;

            while (p) {
                int j;

                next_p = p->next;
                for (j=0; j<p->count; j++) {
                    if (p->storage[j]) {
                        if (resource_types_table && !resource_types_table[j].done && resource_types_table[j].dtor) {
                            resource_types_table[j].dtor(p->storage[j], &p->storage);
                        }
                        free(p->storage[j]);
                    }
                }
                free(p->storage);
                free(p);
                p = next_p;
            }
        }
        free(tsrm_tls_table);
        tsrm_tls_table = NULL;
    }
    // Release resources table
    if (resource_types_table) {
        free(resource_types_table);
        resource_types_table=NULL;
    }
    // Release Thread Lock
    tsrm_mutex_free(tsmm_mutex);
    tsmm_mutex = NULL;
    TSRM_ERROR((TSRM_ERROR_LEVEL_CORE, "Shutdown TSRM"));
    if (tsrm_error_file!=stderr) {
        fclose(tsrm_error_file);
    }
// Delete Thread Key
#if defined(GNUPTH)
    pth_kill();
#elif defined(PTHREADS)
    pthread_setspecific(tls_key, 0);
    pthread_key_delete(tls_key);
#elif defined(TSRM_WIN32)
    TlsFree(tls_key);
#endif
}

tsrm_shutdown pass through all the existing threads, save the next thread node before freeing the current thread, thus the function invoke the resource destructor and start freeing each element from the global variable storage array. Finally release the tsrm_tls_table, so this way we shutdown the thread-safe memory manager.

Here is a look at Apache sapi TSRM calls during MINIT & MSHUTDOWN phase :

In fact the purpose of calling tsrm_startup with 1 as value for expected threads, is that in most cases PHP still used with single-threaded environments so it’s a good reason not to waste memory space.

So by this we come up to the end of this paper.

Stay tuned for more interesting stuff.

References :

• TSRM source code in PHP interpreter repository :   TSRM 
• Extending & Embedding PHP Book.

About

Follow me on twitter : https://twitter.com/r007hunt