| Table of Contents |
|---|
Key Points
- remote RPC using gossip protocol
References
...
| Table of Contents |
|---|
Key Points
- remote RPC using gossip protocol
References
Key Concepts
What is gRPC and Protobuf?
...
This document introduces you to gRPC and protocol buffers. gRPC can use protocol buffers as both its Interface Definition Language (IDL) and as its underlying message interchange format. If you’re new to gRPC and/or protocol buffers, read this! If you just want to dive in and see gRPC in action first, see our Quick Starts.
https://grpc.io/docs/what-is-grpc/faq/
gRPC is a modern, open source remote procedure call (RPC) framework that can run anywhere. It enables client and server applications to communicate transparently, and makes it easier to build connected systems.
gRPC uses HTTP/2, which multiplexes multiple calls on a single TCP connection. All gRPC calls over that connection go to one endpoint.
The main usage scenarios:
- Low latency, highly scalable, distributed systems.
- Developing mobile clients which are communicating to a cloud server.
- Designing a new protocol that needs to be accurate, efficient and language independent.
- Layered design to enable extension eg. authentication, load balancing, logging and monitoring etc.
Overview
In gRPC a client application can directly call methods on a server application on a different machine as if it was a local object, making it easier for you to create distributed applications and services. As in many RPC systems, gRPC is based around the idea of defining a service, specifying the methods that can be called remotely with their parameters and return types. On the server side, the server implements this interface and runs a gRPC server to handle client calls. On the client side, the client has a stub (referred to as just a client in some languages) that provides the same methods as the server.
...
The first step when working with protocol buffers is to define the structure for the data you want to serialize in a proto file: this is an ordinary text file with a .proto extension. Protocol buffer data is structured as messages, where each message is a small logical record of information containing a series of name-value pairs called fields. Here’s a simple example:
message Person {
string name = 1;
int32 id = 2;
bool has_ponycopter = 3;
}Then, once you’ve specified your data structures, you use the protocol buffer compiler protoc to generate data access classes in your preferred language(s) from your proto definition. These provide simple accessors for each field (like name() and set_name()) as well as methods to serialize/parse the whole structure to/from raw bytes – so, for instance, if your chosen language is C++, running the compiler on the above example will generate a class called Person. You can then use this class in your application to populate, serialize, and retrieve Person protocol buffer messages.
As you’ll see in more detail in our examples, you define gRPC services in ordinary proto files, with RPC method parameters and return types specified as protocol buffer messages:
// The greeter service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}gRPC also uses protoc with a special gRPC plugin to generate code from your proto file. However, with the gRPC plugin, you get generated gRPC client and server code, as well as the regular protocol buffer code for populating, serializing, and retrieving your message types. We’ll look at this example in more detail below.
...
gRPC Concepts
https://grpc.io/docs/guides/concepts/
This document introduces some key gRPC concepts with an overview of gRPC’s architecture and RPC life cycle.
It assumes that you’ve read What is gRPC?. For language-specific details, see the Quick Start, tutorial, and reference documentation for your chosen language(s), where available (complete reference docs are coming soon).
Overview
Service definition
Like many RPC systems, gRPC is based around the idea of defining a service, specifying the methods that can be called remotely with their parameters and return types. By default, gRPC uses protocol buffers as the Interface Definition Language (IDL) for describing both the service interface and the structure of the payload messages. It is possible to use other alternatives if desired.
service HelloService {
rpc SayHello (HelloRequest) returns (HelloResponse);
}
message HelloRequest {
string greeting = 1;
}
message HelloResponse {
string reply = 1;
}gRPC lets you define four kinds of service method:
- Unary RPCs where the client sends a single request to the server and gets a single response back, just like a normal function call.
rpc SayHello(HelloRequest) returns (HelloResponse){
}- Server streaming RPCs where the client sends a request to the server and gets a stream to read a sequence of messages back. The client reads from the returned stream until there are no more messages. gRPC guarantees message ordering within an individual RPC call.
rpc LotsOfReplies(HelloRequest) returns (stream HelloResponse){
}- Client streaming RPCs where the client writes a sequence of messages and sends them to the server, again using a provided stream. Once the client has finished writing the messages, it waits for the server to read them and return its response. Again gRPC guarantees message ordering within an individual RPC call.
rpc LotsOfGreetings(stream HelloRequest) returns (HelloResponse) {
}- Bidirectional streaming RPCs where both sides send a sequence of messages using a read-write stream. The two streams operate independently, so clients and servers can read and write in whatever order they like: for example, the server could wait to receive all the client messages before writing its responses, or it could alternately read a message then write a message, or some other combination of reads and writes. The order of messages in each stream is preserved.
rpc BidiHello(stream HelloRequest) returns (stream HelloResponse){
}We’ll look at the different types of RPC in more detail in the RPC life cycle section below.
Using the API surface
Starting from a service definition in a .proto file, gRPC provides protocol buffer compiler plugins that generate client- and server-side code. gRPC users typically call these APIs on the client side and implement the corresponding API on the server side.
- On the server side, the server implements the methods declared by the service and runs a gRPC server to handle client calls. The gRPC infrastructure decodes incoming requests, executes service methods, and encodes service responses.
- On the client side, the client has a local object known as stub (for some languages, the preferred term is client) that implements the same methods as the service. The client can then just call those methods on the local object, wrapping the parameters for the call in the appropriate protocol buffer message type - gRPC looks after sending the request(s) to the server and returning the server’s protocol buffer response(s).
Synchronous vs. asynchronous
Synchronous RPC calls that block until a response arrives from the server are the closest approximation to the abstraction of a procedure call that RPC aspires to. On the other hand, networks are inherently asynchronous and in many scenarios it’s useful to be able to start RPCs without blocking the current thread.
The gRPC programming surface in most languages comes in both synchronous and asynchronous flavors. You can find out more in each language’s tutorial and reference documentation (complete reference docs are coming soon).
RPC life cycle
Now let’s take a closer look at what happens when a gRPC client calls a gRPC server method. We won’t look at implementation details, you can find out more about these in our language-specific pages.
Unary RPC
First let’s look at the simplest type of RPC, where the client sends a single request and gets back a single response.
- Once the client calls the method on the stub/client object, the server is notified that the RPC has been invoked with the client’s metadata for this call, the method name, and the specified deadline if applicable.
- The server can then either send back its own initial metadata (which must be sent before any response) straight away, or wait for the client’s request message - which happens first is application-specific.
- Once the server has the client’s request message, it does whatever work is necessary to create and populate its response. The response is then returned (if successful) to the client together with status details (status code and optional status message) and optional trailing metadata.
- If the status is OK, the client then gets the response, which completes the call on the client side.
Server streaming RPC
A server-streaming RPC is similar to our simple example, except the server sends back a stream of responses after getting the client’s request message. After sending back all its responses, the server’s status details (status code and optional status message) and optional trailing metadata are sent back to complete on the server side. The client completes once it has all the server’s responses.
Client streaming RPC
A client-streaming RPC is also similar to our simple example, except the client sends a stream of requests to the server instead of a single request. The server sends back a single response, typically but not necessarily after it has received all the client’s requests, along with its status details and optional trailing metadata.
Bidirectional streaming RPC
In a bidirectional streaming RPC, again the call is initiated by the client calling the method and the server receiving the client metadata, method name, and deadline. Again the server can choose to send back its initial metadata or wait for the client to start sending requests.
What happens next depends on the application, as the client and server can read and write in any order - the streams operate completely independently. So, for example, the server could wait until it has received all the client’s messages before writing its responses, or the server and client could “ping-pong”: the server gets a request, then sends back a response, then the client sends another request based on the response, and so on.
Deadlines/Timeouts
gRPC allows clients to specify how long they are willing to wait for an RPC to complete before the RPC is terminated with the error DEADLINE_EXCEEDED. On the server side, the server can query to see if a particular RPC has timed out, or how much time is left to complete the RPC.
How the deadline or timeout is specified varies from language to language - for example, not all languages have a default deadline, some language APIs work in terms of a deadline (a fixed point in time), and some language APIs work in terms of timeouts (durations of time).
RPC termination
In gRPC, both the client and server make independent and local determinations of the success of the call, and their conclusions may not match. This means that, for example, you could have an RPC that finishes successfully on the server side (“I have sent all my responses!”) but fails on the client side (“The responses arrived after my deadline!“). It’s also possible for a server to decide to complete before a client has sent all its requests.
Cancelling RPCs
Either the client or the server can cancel an RPC at any time. A cancellation terminates the RPC immediately so that no further work is done. It is not an “undo”: changes made before the cancellation will not be rolled back.
Metadata
Metadata is information about a particular RPC call (such as authentication details) in the form of a list of key-value pairs, where the keys are strings and the values are typically strings (but can be binary data). Metadata is opaque to gRPC itself - it lets the client provide information associated with the call to the server and vice versa.
Access to metadata is language-dependent.
Channels
A gRPC channel provides a connection to a gRPC server on a specified host and port and is used when creating a client stub (or just “client” in some languages). Clients can specify channel arguments to modify gRPC’s default behaviour, such as switching on and off message compression. A channel has state, including connected and idle.
How gRPC deals with closing down channels is language-dependent. Some languages also permit querying channel state.
...
Now let’s look at how Credentials work with one of our supported auth mechanisms. This is the simplest authentication scenario, where a client just wants to authenticate the server and encrypt all data. The example is in C++, but the API is similar for all languages: you can see how to enable SSL/TLS in more languages in our Examples section below.
// Create a default SSL ChannelCredentials object.
auto channel_creds = grpc::SslCredentials(grpc::SslCredentialsOptions());
// Create a channel using the credentials created in the previous step.
auto channel = grpc::CreateChannel(server_name, channel_creds);
// Create a stub on the channel.
std::unique_ptr<Greeter::Stub> stub(Greeter::NewStub(channel));
// Make actual RPC calls on the stub.
grpc::Status s = stub->sayHello(&context, *request, response);
For advanced use cases such as modifying the root CA or using client certs, the corresponding options can be set in the SslCredentialsOptions parameter passed to the factory method.
...
gRPC applications can use a simple API to create a credential that works for authentication with Google in various deployment scenarios. Again, our example is in C++ but you can find examples in other languages in our Examples section.
auto creds = grpc::GoogleDefaultCredentials();
// Create a channel, stub and make RPC calls (same as in the previous example)
auto channel = grpc::CreateChannel(server_name, creds);
std::unique_ptr<Greeter::Stub> stub(Greeter::NewStub(channel));
grpc::Status s = stub->sayHello(&context, *request, response);
This channel credentials object works for applications using Service Accounts as well as for applications running in Google Compute Engine (GCE). In the former case, the service account’s private keys are loaded from the file named in the environment variable GOOGLE_APPLICATION_CREDENTIALS. The keys are used to generate bearer tokens that are attached to each outgoing RPC on the corresponding channel.
...
Here is example of a simple credentials plugin which sets an authentication ticket in a custom header.
class MyCustomAuthenticator : public grpc::MetadataCredentialsPlugin {
public:
MyCustomAuthenticator(const grpc::string& ticket) : ticket_(ticket) {}
grpc::Status GetMetadata(
grpc::string_ref service_url, grpc::string_ref method_name,
const grpc::AuthContext& channel_auth_context,
std::multimap<grpc::string, grpc::string>* metadata) override {
metadata->insert(std::make_pair("x-custom-auth-ticket", ticket_));
return grpc::Status::OK;
}
private:
grpc::string ticket_;
};
auto call_creds = grpc::MetadataCredentialsFromPlugin(
std::unique_ptr<grpc::MetadataCredentialsPlugin>(
new MyCustomAuthenticator("super-secret-ticket")));
A deeper integration can be achieved by plugging in a gRPC credentials implementation at the core level. gRPC internals also allow switching out SSL/TLS with other encryption mechanisms.
...
stub = Helloworld::Greeter::Stub.new('localhost:50051', :this_channel_is_insecure)
...With server authentication SSL/TLS
...
Base case - no encryption or authentication
auto channel = grpc::CreateChannel("localhost:50051", InsecureChannelCredentials());
std::unique_ptr<Greeter::Stub> stub(Greeter::NewStub(channel));
...
With server authentication SSL/TLS
auto channel_creds = grpc::SslCredentials(grpc::SslCredentialsOptions());
auto channel = grpc::CreateChannel("myservice.example.com", channel_creds);
std::unique_ptr<Greeter::Stub> stub(Greeter::NewStub(channel));
...
Authenticate with Google
auto creds = grpc::GoogleDefaultCredentials();
auto channel = grpc::CreateChannel("greeter.googleapis.com", creds);
std::unique_ptr<Greeter::Stub> stub(Greeter::NewStub(channel));
...
C#
Base case - no encryption or authentication
var channel = new Channel("localhost:50051", ChannelCredentials.Insecure);
var client = new Greeter.GreeterClient(channel);
...
With server authentication SSL/TLS
var channelCredentials = new SslCredentials(File.ReadAllText("roots.pem")); // Load a custom roots file.
var channel = new Channel("myservice.example.com", channelCredentials);
var client = new Greeter.GreeterClient(channel);
Authenticate with Google
using Grpc.Auth; // from Grpc.Auth NuGet package
...
// Loads Google Application Default Credentials with publicly trusted roots.
var channelCredentials = await GoogleGrpcCredentials.GetApplicationDefaultAsync();
var channel = new Channel("greeter.googleapis.com", channelCredentials);
var client = new Greeter.GreeterClient(channel);
...
Authenticate a single RPC call
var channel = new Channel("greeter.googleapis.com", new SslCredentials()); // Use publicly trusted roots.
var client = new Greeter.GreeterClient(channel);
...
var googleCredential = await GoogleCredential.GetApplicationDefaultAsync();
var result = client.SayHello(request, new CallOptions(credentials: googleCredential.ToCallCredentials()));
...
...
Base case - No encryption or authentication
import grpc
import helloworld_pb2
channel = grpc.insecure_channel('localhost:50051')
stub = helloworld_pb2.GreeterStub(channel)With server authentication SSL/TLS
Client:
import grpc
import helloworld_pb2
with open('roots.pem', 'rb') as f:
creds = grpc.ssl_channel_credentials(f.read())
channel = grpc.secure_channel('myservice.example.com:443', creds)
stub = helloworld_pb2.GreeterStub(channel)Server:
import grpc
import helloworld_pb2
from concurrent import futures
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
with open('key.pem', 'rb') as f:
private_key = f.read()
with open('chain.pem', 'rb') as f:
certificate_chain = f.read()
server_credentials = grpc.ssl_server_credentials( ( (private_key, certificate_chain), ) )
# Adding GreeterServicer to server omitted
server.add_secure_port('myservice.example.com:443', server_credentials)
server.start()
# Server sleep omittedAuthenticate with Google using a JWT
import grpc
import helloworld_pb2
from google import auth as google_auth
from google.auth import jwt as google_auth_jwt
from google.auth.transport import grpc as google_auth_transport_grpc
credentials, _ = google_auth.default()
jwt_creds = google_auth_jwt.OnDemandCredentials.from_signing_credentials(
credentials)
channel = google_auth_transport_grpc.secure_authorized_channel(
jwt_creds, None, 'greeter.googleapis.com:443')
stub = helloworld_pb2.GreeterStub(channel)Authenticate with Google using an Oauth2 token
import grpc
import helloworld_pb2
from google import auth as google_auth
from google.auth.transport import grpc as google_auth_transport_grpc
from google.auth.transport import requests as google_auth_transport_requests
credentials, _ = google_auth.default(scopes=(scope,))
request = google_auth_transport_requests.Request()
channel = google_auth_transport_grpc.secure_authorized_channel(
credentials, request, 'greeter.googleapis.com:443')
stub = helloworld_pb2.GreeterStub(channel)Java
Base case - no encryption or authentication
ManagedChannel channel = ManagedChannelBuilder.forAddress("localhost", 50051)
.usePlaintext(true)
.build();
GreeterGrpc.GreeterStub stub = GreeterGrpc.newStub(channel);With server authentication SSL/TLS
...
To enable TLS on a server, a certificate chain and private key need to be specified in PEM format. Such private key should not be using a password. The order of certificates in the chain matters: more specifically, the certificate at the top has to be the host CA, while the one at the very bottom has to be the root CA. The standard TLS port is 443, but we use 8443 below to avoid needing extra permissions from the OS.
Server server = ServerBuilder.forPort(8443)
// Enable TLS
.useTransportSecurity(certChainFile, privateKeyFile)
.addService(TestServiceGrpc.bindService(serviceImplementation))
.build();
server.start();If the issuing certificate authority is not known to the client then a properly configured SslContext or SSLSocketFactory should be provided to the NettyChannelBuilder or OkHttpChannelBuilder, respectively.
On the client side, server authentication with SSL/TLS looks like this:
// With server authentication SSL/TLS
ManagedChannel channel = ManagedChannelBuilder.forAddress("myservice.example.com", 443)
.build();
GreeterGrpc.GreeterStub stub = GreeterGrpc.newStub(channel);
// With server authentication SSL/TLS; custom CA root certificates; not on Android
ManagedChannel channel = NettyChannelBuilder.forAddress("myservice.example.com", 443)
.sslContext(GrpcSslContexts.forClient().trustManager(new File("roots.pem")).build())
.build();
GreeterGrpc.GreeterStub stub = GreeterGrpc.newStub(channel);Authenticate with Google
The following code snippet shows how you can call the Google Cloud PubSub API using gRPC with a service account. The credentials are loaded from a key stored in a well-known location or by detecting that the application is running in an environment that can provide one automatically, e.g. Google Compute Engine. While this example is specific to Google and its services, similar patterns can be followed for other service providers.
GoogleCredentials creds = GoogleCredentials.getApplicationDefault();
ManagedChannel channel = ManagedChannelBuilder.forTarget("greeter.googleapis.com")
.build();
GreeterGrpc.GreeterStub stub = GreeterGrpc.newStub(channel)
.withCallCredentials(MoreCallCredentials.from(creds));Node.js
Base case - No encryption/authentication
...
Base case - no encryption or authentication
final channel = new ClientChannel('localhost',
port: 50051,
options: const ChannelOptions(
credentials: const ChannelCredentials.insecure()));
final stub = new GreeterClient(channel);With server authentication SSL/TLS
// Load a custom roots file.
final trustedRoot = new File('roots.pem').readAsBytesSync();
final channelCredentials =
new ChannelCredentials.secure(certificates: trustedRoot);
final channelOptions = new ChannelOptions(credentials: channelCredentials);
final channel = new ClientChannel('myservice.example.com',
options: channelOptions);
final client = new GreeterClient(channel);Authenticate with Google
// Uses publicly trusted roots by default.
final channel = new ClientChannel('greeter.googleapis.com');
final serviceAccountJson =
new File('service-account.json').readAsStringSync();
final credentials = new JwtServiceAccountAuthenticator(serviceAccountJson);
final client =
new GreeterClient(channel, options: credentials.toCallOptions);Authenticate a single RPC call
// Uses publicly trusted roots by default.
final channel = new ClientChannel('greeter.googleapis.com');
final client = new GreeterClient(channel);
...
final serviceAccountJson =
new File('service-account.json').readAsStringSync();
final credentials = new JwtServiceAccountAuthenticator(serviceAccountJson);
final response =
await client.sayHello(request, options: credentials.toCallOptions);...