Twirp simplifies service design when compared with a REST endpoint: method definitions, message types and parsing is handled by the framework (i.e. you don’t have to worry about JSON fields or types). However, there are still some things to consider when making a new service in Twirp, mainly to keep consistency.
The recommended folder/package structure for your twirp
/cmd /<service> main.go /rpc /<service> service.proto // and auto-generated files /internal /<service>server server.go // and usually one other file per method
For example, for the Haberdasher service it would be:
/cmd /haberdasherserver main.go /rpc /haberdasher service.proto service.pb.go service.twirp.go /internal /haberdasherserver server_test.go server.go make_hat_test.go make_hat.go
- Keep the
.protoand generated files in their own package.
- Do not implement the server or other files in the same package. This allows other services to do a "clean import" of the autogenerated client.
- Do not name the package something generic like
service; name it after your service. Remember that the package is going to be imported by other projects that will likely import other clients from other services as well.
.proto file is the source of truth for your service design.
- The first step to design your service is to write a
.protofile. Use that to discuss design with your coworkers before starting the implementation.
- Use proto3 (first line should be
syntax="proto3"). Do not use proto2.
option go_package = "<service>";for the Go package name.
- Add comments on message fields; they translate to the generated Go interfaces.
- Don’t worry about fields like
user_idbeing auto-converted into
UserIdin Go. I know, the right case should be
UserID, but it's not your fault how the protoc-gen-go compiler decides to translate it. Avoid doing hacks like naming it
user_i_dso it looks "good" in Go (
- rpc methods should clearly be named with
(i.e.: ListBooks, GetBook, CreateBook, UpdateBook, RenameBook, DeleteBook). See more in "Naming Conventions" below.
The header of the
.proto file should look like this (change
syntax = "proto3" package <organization>.<repo>.<service>; option go_package = "<service>";
Specifying protoc version and using retool for protoc-gen-go and protoc-gen-twirp
Code generation depends on
protoc and its plugins
protoc-gen-twirp. Having different versions may cause problems.
Make sure to specify the required
protoc version in your README or
For the plugins, you can use retool. Like with most Go commands used to manage your source code, retool makes it easy to lock versions for all team members.
$ retool add github.com/golang/protobuf/protoc-gen-go master $ retool add github.com/twitchtv/twirp/protoc-gen-twirp master
Using a Makefile is a good way to simplify code generation:
gen: # Auto-generate code retool do protoc --proto_path=. --twirp_out=. --go_out=. rpc/<service>/service.proto upgrade: # Upgrade glide dependencies retool do glide update --strip-vendor retool do glide-vc --only-code --no-tests --keep '**/*.proto'
Like in any other API or interface, it is very important to have names that are simple, intuitive and consistent.
Respect the Protocol Buffers Style Guide:
underscore_separated_namesfor field names.
CAPITALS_WITH_UNDERSCORESfor enum value names.
For naming conventions, the Google Cloud Platform design guides are a good reference:
- Use the same name for the same concept, even across APIs.
- Avoid name overloading. Use different names for different concepts.
- Include units on field names for durations and quantities (e.g.
delay_secondsis better than just
For times, we have a few Twitch-specific conventions that have worked for us:
- Timestamp names should end with
_atwhenever possible (i.e.
- Timestamps should be RFC3339 strings
(in Go it's very easy to generate these with
t.Format(time.RFC3339)and parse them with
- Timestamps can also be a
google.protobuf.Timestamp, in which case their names should end with
Default Values and Required Fields
In proto3 all fields have zero-value defaults (string is
"", int32 is
all fields are optional.
If you want to make a required field (i.e. "name is required"), it needs to be
handled by the service implementation. But to make this clear in the
- Add a "required" comment on the field. For example
string name = 1; // requiredimplies that the server implementation will return an
twirp.RequiredArgumentError("name")if the name is empty.
If you need a different default (e.g. limit default 20 for paginated
collections), it needs to be handled by the service implementation. But to make
this clear in the
- Add a "(default X)" comment on the field. For example
int32 limit = 1; // (default 20)implies that the server implementation will convert the zero-value 0 to 20 (0 == 20).
- For enums, the first item is the default.
Your service implementation cannot tell the difference between empty and missing
fields (this is by design). If you really need to tell them apart, you need to
use an extra bool field, or use
google/protobuf.wrappers.proto messages (which
can be nil in go).
Protocol Buffers do not specify errors. You can always add an extra field on the returned message for the error, but Twirp has an excellent system that you should use instead:
- Familiarize yourself with the possible Twirp error codes and use
the ones that make sense for each situation (i.e.
Internal). The codes are very straightforward and are almost the same as in gRPC.
- Always return a
twirp.Error. Twirp allows you to return a regular
error, that will get wrapped with
twirp.InternalErrorWith(err), but it is better if you explicitly wrap it yourself. Being explicit makes the server and the client to always return the same twirp errors, which is more predictable and easier for unit tests.
- Include possible errors on the
.protofile (add comments to RPC methods).
- But there's no need to document all the obvious
Internalerrors, which can always happen for diverse reasons (e.g. backend service is down, or there was a problem on the client).
- Make sure to document (with comments) possible validation errors on the
specific message fields. For example
int32 amount = 1; // must be positiveimplies that the server implementation will return a
twirp.InvalidArgumentError("amount", "must be positive")error if the condition is not met.
- Required fields are also validation errors. For example, if a given string
field cannot be empty, you should add a "required" comment in the proto file,
which implies that a
twirp.RequiredArgumentError(field)will be returned if the field is empty (or missing, which is the same thing in proto3). If you are using proto2 (I hope not), the "required" comment is still preferred over the required field type.