Implementing Database Connection Pool and Managing Concurrency with OpenSwoole GRPC

The OpenSwoole GRPC feature was newly introduced in version 22.0.0, enabling you to construct GRPC services using OpenSwoole and PHP.

How to use OpenSwoole GRPC to implement database connection pool

Implement database connection pool and manage concurrency with OpenSwoole GRPC

To manage concurrency and implement a database connection pool, OpenSwoole GRPC can be utilized. When separating certain features and services into independent microservices, GRPC can function as an internal communication protocol within your application.

Due to the process model of PHP-FPM, MySQL databases may become easily overloaded with a large number of connections from multiple PHP-FPM processes, each with one connection. This is especially true when I/O is a bottleneck.

With OpenSwoole GRPC, a data layer service can be constructed to control concurrency to databases by establishing a limited number of connections to protect the data layer, while simultaneously serving connections from a large number of PHP-FPM processes. By limiting the number of concurrent connections, your database will not become overloaded.

Here is an example of how to create a connection pool in OpenSwoole 22:

$mysql_pool = new ClientPool(PDOClientFactory::class, new PDOConfig(), 8, true);

After creating the connection pool, you can use the connections within it in your OpenSwoole GRPC services, as shown below:

$mysqlClient = $mysqlPool->get();
$mysqlClient->query('SELECT SLEEP(10)')->fetch();
$mysqlPool->put($mysqlClient);

By limiting the number of connections to the database to 8 per OpenSwoole GRPC process, you can control the connections on the database.

Once the GRPC service is set up, you can use any GRPC client to send requests to the service and communicate with your data layer. This includes using a GRPC client in PHP-FPM.

Here are the steps to build GRPC services using OpenSwoole and PHP:

1. To install the OpenSwoole extension, execute the following command:

#!/bin/bash

# Install git to access the source code from GitHub
$ sudo apt install git

$ cd /tmp && git clone https://github.com/openswoole/ext-openswoole.git && \
    cd ext-openswoole && \
    git checkout v22.1.2 && \
    phpize  && \
    ./configure --enable-openssl \
                --enable-mysqlnd \
                --enable-sockets \
                --enable-http2 \
                --enable-hook-curl \
                --with-postgres \
    sudo make && sudo make install

Alternatively, you can refer to https://openswoole.com/docs/get-started/installation for other installation methods.

Once the extension is installed, add the line extension=openswoole.so to the end of your php.ini file, or create a new ini file in the conf.d folder to enable OpenSwoole.

To verify that OpenSwoole has been successfully enabled, execute the command php --ri openswoole.

~ php --ri openswoole

openswoole

Open Swoole => enabled
Author => Open Swoole Group <[email protected]>
Version => 22.0.0-dev
Built => Dec  7 2022 19:29:43
coroutine => enabled with boost asm context
kqueue => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 3.0.5 5 Jul 2022
dtls => enabled
http2 => enabled
curl-native => enabled
pcre => enabled
zlib => 1.2.11
mysqlnd => enabled
postgresql => enabled

Directive => Local Value => Master Value
openswoole.enable_coroutine => On => On
openswoole.enable_preemptive_scheduler => Off => Off
openswoole.display_errors => On => On
openswoole.unixsock_buffer_size => 262144 => 262144

2. Setting up Composer and Installing OpenSwoole GRPC Library

Create a new folder and generate a composer.json file using the following command:

composer init

Once the composer.json file is created, install the OpenSwoole GRPC package and Google Protocol Buffers using the following commands:

composer require openswoole/grpc
composer require google/protobuf

Ensure that your composer.json file looks similar to the following:

{
    "name": "openswoole-grpc/hello-world",
    "require": {
        "openswoole/grpc": "^22.0",
        "google/protobuf": "^3.21"
    }
}

Generate the autoload.php file using the following command:

composer dump-autoload

3. Defining the Protocol Buffers Protocol

To define the gRPC service and its method request and response types, you will use Protocol Buffers, which is a language- and platform-neutral data serialization format developed by Google.

1. Create a new file called greetings.proto.

2. Add the following line to the top of the file to specify that you will be using proto3 in this example:

syntax = "proto3";

3. Declare the package for your service:

package helloWorld;

4. Define a service named Greeter in the greetings.proto file:

service Greeter {
  ...
}

5. Create the message type for the response message, which in this case is HelloReply:

message HelloReply {
    string message = 1;
}

6. Define the request message type for the SayHello method of the Greeter service:

message HelloRequest {
    string name = 1;
}

7. Add the SayHello method to the Greeter service, which receives a HelloRequest and returns a HelloReply:

service Greeter {
    // Sends a greeting
    rpc SayHello (HelloRequest) returns (HelloReply) {}
}

8. Your completed greetings.proto file should look like this:

syntax = "proto3";

package helloWorld;

// The greeting service definition.
service Greeter {
    // Sends a greeting
    rpc SayHello (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
    string name = 1;
}

// The response message containing the greetings
message HelloReply {
    string message = 1;
}

4. Implementing the gRPC service logic

Download the openswoole-grpc code generator plugin.

## For Apple M1
wget https://github.com/openswoole/protoc-gen-openswoole-grpc/releases/download/0.1.1/protoc-gen-openswoole-grpc-0.1.1-darwin-arm64.tar.gz

## You can find version for another platform on `https://github.com/openswoole/protoc-gen-openswoole-grpc/releases`

## Extract the file
mkdir protoc-gen-openswoole-grpc-0.1.1 && tar -zxvf protoc-gen-openswoole-grpc-0.1.1-darwin-arm64.tar.gz --directory ./protoc-gen-openswoole-grpc-0.1.1/

## Copy to /usr/local/bin/
sudo cp ./protoc-gen-openswoole-grpc-0.1.1/protoc-gen-openswoole-grpc /usr/local/bin/

Generate the stub codes with commands:

mkdir src

protoc --php_out=./src --openswoole-grpc_out=./src helloworld.proto

If you encounter the warning "protoc-gen-openswoole-grpc" can’t be opened because Apple cannot check it for malicious software on your Mac, you can follow these steps to allow the program to run:

1. Go to System Preferences > Security & Privacy.
2. Click on the lock icon at the bottom left to make changes.
3. Enter your password if prompted.
4. Under the "General" tab, you should see a message that says "protoc-gen-openswoole-grpc was blocked from use because it is not from an identified developer." Click on the "Allow Anyway" button next to it.
5. You should now be able to run the protoc command without any issues.

To execute the script again, simply click on it.

If you want to use the generated files, you can include the following section in your composer.json file.

"autoload": {
    "psr-4": {
      "": "src"
    }
}

Next, you can incorporate the service logic into the generated PHP service files, using the Helloworld GreeterService example as a guide:

<?php declare(strict_types=1);

namespace HelloWorld;

use OpenSwoole\GRPC;

class GreeterService implements GreeterInterface
{
    /**
    * @param GRPC\ContextInterface $ctx
    * @param HelloRequest $request
    * @return HelloReply
    *
    * @throws GRPC\Exception\InvokeException
    */
    public function SayHello(GRPC\ContextInterface $ctx, HelloRequest $request): HelloReply
    {
      // your code
    }
}

Create the function implementation, for example:

public function SayHello(GRPC\ContextInterface $ctx, HelloRequest $request): HelloReply
{
    $name = $request->getName();
    $out  = new HelloReply();
    $out->setMessage('hello ' . $name);
    return $out;
}

5. Instantiate the OpenSwoole GRPC server

Example

<?php

use Helloworld\GreeterService;
use Helloworld\StreamService;
use OpenSwoole\GRPC\Middleware\LoggingMiddleware;
use OpenSwoole\GRPC\Middleware\TraceMiddleware;
use OpenSwoole\GRPC\Server;

(new Server('127.0.0.1', 9501))
    ->register(GreeterService::class)
    ->register(StreamService::class)
    ->withWorkerContext('worker_start_time', function () {
        return time();
    })
    // use middlewares
    ->addMiddleware(new LoggingMiddleware())
    ->addMiddleware(new TraceMiddleware())
    ->set([
        'log_level' => \OpenSwoole\Constant::LOG_INFO,
    ])
    ->start();

6. Utilize the OpenSwoole GRPC client to send requests to the GRPC service

Example:

<?php
declare(strict_types=1);

use Helloworld\HelloRequest;
use OpenSwoole\Constant;
use OpenSwoole\GRPC\Client;

co::run(function () {
  $conn    = (new Client('127.0.0.1', 9501))->connect();
  $client  = new Helloworld\GreeterClient($conn);
  $message = new HelloRequest();
  $message->setName(str_repeat('x', 10));
  $out = $client->sayHello($message);
  var_dump($out->serializeToJsonString());
  $conn->close();
  echo "closed\n";
}

7. Use the PHP-FPM GRPC client to send requests to the GRPC service

7.1 Installing the GRPC extension

pecl install grpc

7.2 Installing the dependencies for PHP-FPM GRPC client

composer require google/protobuf
composer require grpc/grpc

7.3 Installing grpc_php_plugin

brew install grpc

7.4 Create client stub code by using the PHP-FPM Protocol Buffer Compiler

protoc --php_out=./src \ 
--grpc_out=./src \
--plugin=protoc-gen-grpc=grpc_php_plugin \
helloworld.proto

7.5 Create and use PHP-FPM GRPC client in your PHP-FPM application

<?php

require './vendor/autoload.php';

use Grpc\ChannelCredentials;
use Helloworld\HelloRequest;

$client = new Helloworld\GreeterClient('127.0.0.1:9501', [
    'credentials' => ChannelCredentials::createInsecure(),
]);

$message = new HelloRequest();

$message->setName(substr(str_shuffle('abcdefghijklmnopqrstuvwxyz'), 0, 8));

list($response, $status) = $client->sayHello($message)->wait();
if ($status->code !== Grpc\STATUS_OK) {
    echo "ERROR: " . $status->code . ", " . $status->details . PHP_EOL;
    exit(1);
}
echo $response->serializeToJsonString() . PHP_EOL;

References:

• https://github.com/openswoole/openswoole/tree/master/grpc
• https://openswoole.com/docs/grpc