A Basic gRPC Example

Background

Many infrastructure developers have experienced this tedium: you have some functionality on one host you would like to access on another. For a while now, language runtimes have shipped with some sort of remote access mechanism. For Go, the gob encoding library used in conjunction with net/rpc as a transport mechanism was a suggested solution. By time Go arrived on the scene, most companies were operating heterogenous environments so limiting distributed functionality simply to Go wasn't realistic.

Dealing with heterogenous environments requires the adoption of WAN-friendly technologies that can be adapted to the LAN with a little bit of effort. Traditionally, this meant the developer would:

• Define a WAN-friendly message format in JSON that would encapsulate the functionality.
• Build and operate a webservice to translate that JSON into local function calls.
• Possibly introduce a bespoke type system to express how parameters should be cast.

Large companies like Google and Facebook had long wrestled with the first challenge - defining a WAN-friendly message format. Protocol buffers was created to meet this need. From there, it became obvious that it should be possible to automatically generate both servers and clients to bind messages to local function calls directly. This is the motivation for gRPC.

Why gRPC Is Worth Considering

Building complex APIs with web technologies is achievable but painful, and gRPC addresses many of the pain points developers hit again and again:

• Documentation. Developers of HTTP APIs have options ranging from API Blueprint to Swagger to RAML and others. None of these has ever gained wide adoption. Because documentation formatted with these tools is not fundamental to the operation of a service, bitrot often results. How gRPC fixes this: gRPC proto files are fundamental to development and are high-level enough to be treated as documentation.
• Bidirectionality. Bidirectional communication for HTTP APIs is a longstanding problem with no single solution. Alternatives such as Websockets must sit alongside traditional ReST API methods. How gRPC fixes this: gRPC supports bidirectional communication through streams as a core feature.
• Testing. HTTP APIs can only be hardened through iterative unit testing. How gRPC fixes this: we can use the features of the host implementation langage for more effective testing. In the case of Go, the type system and testing infrastructure are more directly available.
• Errors. HTTP has a limited set of error codes. How gRPC fixes this: gRPC binds native types to services, so we can integrate native error handling.

HTTP APIs are still necessary to establish browser communication (there are attempts at getting browsers to speak directly to gRPC services but these seem quite hacky). At this point my personal approach is to create the thinnest HTTP layer possible with gRPC services doing most of the interesting work behind the scenes.

A Simple Example: Counting

I'll illustrate how gRPC works with Go by creating a very simple sample project - a server that offers two functions:

• increment(key,value) which adds value to a counter identified by key, with a previously unseen key assumed to have a value of 0.
• read(key) return the value for key, or an error if key is not defined.

Prerequisites

• Install the protocol buffers compiler. I use Debian and it is available with this command: apt-get install protobuf-compiler. This tool translates protocol buffers files into stub definitions in Go that we will satisfy with our own functionality.
• Install grpc: go get -u google.golang.org/grpc. These libraries are used to bind your functionality to generated gRPC code.
• Download sample code repo: git clone https://github.com/bradclawsie/grpc-basics

Step One: Define Messages

We've already defined the spec for our API above. The first step is to build a protocol buffers definition of the input, output and rpc types that gRPC will use to generate stub code.

counter.proto

For this example, we only care about Go and will only generate stub definitions for Go, but keep in mind that the primary value of gRPC is that it will generate native messages and service bindings in any of the supported languages.

If you have followed the instructions in the prerequisites, you should be able to generate the Go stub by typing: make protoc in the counter directory.

The resulting file counter/counter.pb.go should be generated and look like this. Even though this file looks a bit messy, review it. Note how there are native Go type definitions for your protocol buffer message types, along with getters to extract values. I recommend making as little use of these generated types as possible. The first thing your rpc endpoints should do is to extract values using the getters and put them into instances of types that you control and test.

Also note the use of Go interfaces in the generated code, namely CounterClient and CounterServer. Ultimately, it is up to you to decide how the internal implementations of these endpoints work. All Go requires is that you satisfy the interface. This is a nice feature of Go that makes working with gRPC more straightforward than in other languages that don't have the notion of compile-time interface satisfaction. Note that the generated functions contain an error return parameter that is idiomatic for Go. Make use of this for application errors and try to avoid embedding custom error values into your protocol buffer messages. The functions you will write that satisfy the gRPC interfaces should be as shallow as possible. I recommend putting domain-specific functionality into private methods that you can test independently of using gRPC. This is done in a contrived manner in the example code to illustrate the point.

Step Two: Write The Server

The CounterServer interface demands two functions be implemented:

• IncrementCounter(ctx context.Context, in *pb.Increment ...) (*pb.ValueResponse, error)
• ReadCounter(ctx context.Context, in *pb.Read ...) (*pb.ValueResponse, error)

Note that as mentioned above, the satisfying implementations only extract embedded values from generated types to call a private function. Note that there is also a generated function RegisterCounterServer that is used to bind our implementation to the gRPC generated server. This is where the CounterServer interface must be satisfied by the server instance s.

Compile this with go build and start it running.

counter_server.go

Step Three: Write The Client

The CounterClient interface offers a calling convention analogous to the methods in CounterServer defines.

• IncrementCounter(ctx context.Context, in *pb.Increment ...) (*pb.ValueResponse, error)
• ReadCounter(ctx context.Context, in *pb.Read ...) (*pb.ValueResponse, error)

Build this with go build, run like:

• Set a key:./counter_client inc key 3
• Increment a key:./counter_client inc key 2
• Read a key:./counter_client get key
• Read a bad key:./counter_client get not-key

counter_client.go

Summary

I hope this was a useful introduction to using gRPC with Go. Generating servers and clients can save you a lot of boilerplate work as long as you treat the resulting generated functions and types as only a thin layer to your own application code. Don't be afraid to read the generated code - this is key to understanding the proper use of gRPC.

last update 2019-09-17