VIDEO: Building API's with Django and GraphQL

Video Transcript:

Quick Disclaimer before we go too much further- hype driven development is definitely not a good thing so don't go back and rewrite your API’s because I said it was awesome in this talk. REST is a great framework and there are a lot of really good tool sets around it.

Where are we going to go today? We are going to talk a little bit about GraphQL what it is, some of the key concepts you need to know and then were going to build and do some live coding, if I can with a mic in my hand - super simple GraphQL api to give you guys a feel for how it works in some real code, going to chat about a few of the pros and cons we’ve experienced running it in production for the past few months and then chat a little but about what's best for Django/GraphQL going forward. There's some cool open source stuff going on.

So what is GraphQL?

GraphQL is a couple things - it’s a query language specification and its a collection of tools designed to operate over a single endpoint via http optimizing for performance and flexibility in development,so its both a query language that defines how you send data back and forth to the server and its a set of tools for building APIs on the backend and tool set for querying those APIS on the frontend and the tools are actually one of the things that makes GraphQL really awesome because there is a lot of really good work going on in those tool sets.

One of the main differences between GraphQL and REST is that as opposed to REST where you define resources which are defined by an ID and then you have a number of URLs that you can access an individual resource or multiple resources or modify those resources based on your HTTP verbs. Everything in GraphQL happens over a single endpoint where you are passing the queries and the server so that's one of the big differences and because it uses a flexible type system it allows you a lot of flexibility when you are developing and also optimizes for performance which we’ll go over in a little bit.

Key features. So instead of REST, where you query for a certain object and you get back all of the objects data based on whatever the server want to send you, in GraphQL you transfer a lot of the data requirements over to the front end so the front end gets to say, I want this piece of data and I want these attributes for this piece of data and that's all that gets sent over the wire - nothing more nothing less - the front end is in more control.

You also can easily request multiple resources in a single request. One of the issues that we ran into when we are running REST API basing our app off of that was that we ran into screens that we were trying to build but we were requesting multiple different types of resources and having a whole bunch of different API requests going on at the same time and we then had to finagle together into one screen and render it. So with GraphQL you can send, you can get all the data you need for any situation with one request because you can really easily compose a cross resources.

The third thing that we definitely have to mention - this is a back end meetup so I won't go to in depth in the front end tooling - but the front end tooling around GraphQL is phenomenal, a lot of the stuff that we used to be doing by hand using a REST API, so doing the fetch requests, normalizing it, storing it in state, querying it from state, and displaying it for the user, a lot of that is handled automatically in a more declarative manner. So I won't go super deep into this but you should definitely check out the front end tooling, especially when using react it's like a match made in Heaven.

GraphQL basically divides your schema into four different types of things. The base model is types, so in REST you have resources which are defined by an ID and you can fetch those and manipulate them by sending different types of requests to different endpoints and the URLS are really closely tied to the resource. In GraphQL they are separate so you define types on one hand and then you find seperate things to interact with those types. At that base everything in GraphQL is a type: your queries are types, your mutations are types, that's kind of where it starts and the base. In our case we define all of our models as types and then we define the queries and mutations it allows us to change them.

The second thing is queries: instead of using your typical get verbs like you do in a REST context, with GraphQL you get to define your own queries and define what types of data returns back and then the front end is in control of what data it wants to query for and how it wants to display it. We’ll get into that in a little but, also mutations, the same thing as post put patch delete on REST except you get to define them and the front end has a little bit more control.

And the last one is subscriptions so, this one really doesn't have an analog in REST but it's one of the things were really excited about with GraphQL is that it has the ability for you to both query data and receive live updates in real time from your backend as that data changes and re render it there, so it kinda ties with the querying and the updating together with makes it really easy to make real time maps.

Graphene is pretty much the go to framework for doing anything GraphQL and Django these days. The team of Graphene is awesome they are doing a great job. The libraries are doing a great job of running things there. This is what we are going to chat through today, this is what we’ve built our API on and it's been pretty solid, we’ve been running it in production.

To give you  a taste of what it's like to run a GraphQL App, we’re going to build a super simple to do app that everybody has seen a million time as example. In this example app we’ve got a todo model which is super simple, ties into a user model, has a title for what needs to be done, and a (?) to say whether it been done or not. Super simple. And we’ll build some queries and mutations on top of this and chat about what this actually looks like on the code level.

So getting set up with Graphene is super simple, again the docs have much more information on this but through changes pretty much, one you just install graphene, then you have to set up the GraphQL view in your URL Config, similar as you would set up any other view,  and then you pass in the schema that you're going to use as the root of you API to that view, and then one of the cool parts of the toolset that GraphQL offer you is this interactive documentation/interactive playground with the API which is super sweet, we pass in this GraphQL true flag which lets us access that and then you gotta create your route schema files, so we create that sas the root of our project. Right now it's got a blind query in there and we’ll build on top of it from there.

So, types, we’ll start out with. Everything is a type in GraphQL even your queries and your mutations, so we’re going to have to define some types to start of. So, we define todo type which matches up with our todo model and graphene is super smart, it’s got a Django integration, you just say, hey, this thing is a Django object, this is the model you are going to use - graphene, behind the scenes will go through all the fields, it basically does the same jobs as a serializer if you are familiar. It does the job of serializing and then passing data to the front end.

Let’s go in here real quick and try a code, it’s gonna be harder on a small screen, into our types module we’ve got the todo type defined here and we also have a user type. One of the really neat things about the way GraphQL handles things, and Graphene in particular as a library, is that any time you define two different object types that are linked in Django models, there is a to do object that has a foreign key to the user model as well Graphene, behind the scenes, because we define these two types, automatically know the relationship there, and lets us query across that relationship, both from any user queries that we create, and from any todo queries that we create. So we can query as deep as we want down through relationships automatically based on just defining these types. In this case because the user model has a lot of stuff we don't’ want people to see, we aren’t wanting to show passes to people, we’ve smoothed the fields. We include only the fields we want you to see.

Queries: we’re going to want a way to actually see the data from our backend and get some responses and show it in our app if we want to. This is, what you are seeing here, is the GraphQL query language, so on your left hand side of the screen there is a GraphQL query syntax, it will look a little odd because there aren’t any colons or quotations marks or anything, but essentially we are defining a query request for a query that we are going to call “all todos” and from that all todos we are going to get a todo type back, and then these three, the ID, title and done, we’re saying we want to query the all todo’s query, we’re going to get a todo type back, and we only want to get back the id, the title and the done field.

We don't want the user object, we only want those three parts, the type that we got, and the data that we get back from this is like this, it will only give us back the id, the title and the done value. If we go back into code for a second, up in the schema, this is our root schema file that we defined earlier, we’re going to, we pass in the root query here to the root schema and then if we go in, we got todos, GraphQL queries, we’re going to create this all todo’s query here.

So, the way you create a query in graphene is you create a class and you subclass the graphene object tag, remember how I said everything is just and object type in GraphQL, query is the same thing. And then what you have to do is, this is actually defining our wrapper object and then to define the actual query you say all todos, which is the query that we want to query from the backend, and then you list the return tags, so you say we wanna get back a list of to do tags as a return value from this query, and then we also can pass in a arguments as like keyword arguments that Graphene list things, this can be any type, graphene has built in strings, lists, fields, all sorts of good stuff, but for now we want to list these back, we want to pass into the parameter.

To reach query define you’ll  have to define the result function which we do down here. We get this info parameter which is pretty much akin to a normal Django request object. We can query the  user stuff with cookies, all that good stuff. And then we get past in this done parameter, this parameter up here, so nothing to special, queries for all the objects, done is set if done is been past that query or not, if so it filters it and then returns the result, it just returns the raw Django models and then behind the scenes Graphene takes care of serializing it, figuring out which field the front end wants and then passes it all the way down.

So if we go over here to our live GraphQL playground this is one of the cool parts that we get access to using that. If I were to define a query here using GraphQL query language, I say all, I want all to do’s, and then from these to do type objects I want the ID, I want the Title, and whether it’s done or not. Fire that off and here we get a list of all the to dos from my database with the title, the id and the done.

If we were recently building an app, this is a requirement, say the requirement has changed and we want to change the user, on the homescreen of our app we want to show which user owns each to do item, it’s pretty simple, we just add user parameter and then we pass in what the id of the user and the first name of the user. Because don’t I know the first name specified in the database, first name will be blank but it allows you to query that data right along side it so you only ever pass which you need across the wire and never anything  more than that. It’s super flexible, when the requirements change its really easy to swap back and forth.

Next one, mutations. So, again this is GraphQL query language, it’s still kind of a foggy syntax but we are essentially creating  a mutation on the back end that's going to modify one of our types. Going to change how its stored in the back end then it's going to return some data back to us and again we have control over what data we get returned from this mutation.

We’re going to create a new mutation that allow us to create a new todo item. Pass the title, pass in the done flag, whether it's done or not, and then we’ll get the data back and then we can render that for user approval if we wanted to on front end. So if we go under mutations here, import graphene, import the type and the model that we need to.

Mutations are a little bit differently defined than queries are - it’s still an object type underneath but its wrapped by this graphene mutation type that we have to subclass, so we subclass that and then instead of defining the mutation here and putting the return value here this whole thing is the return value so we’re going to return an individual to do type from this create to do mutation.

Then in order to take an argument from here we want to take in the title and the done flag arguments, we need to subclass it, pass in the raw graphene types and then much the same way as we did our resolver functioning queries we need to create a mutate function in here as well. So we’re going to pass in info, we get passed in the title that we defined as an argument and the done flag that we defined as an argument. We can pull the user of the current login user, we can pull that off of the context much like you can in REST framework and we created todo object and then this is the foggy part, still not sure why they did this behind the scenes, but you return an instance of the current class and you pass in the todo object that you created as an argument to it.

Kind of some foggy stuff going on there, not really sure why that decision was made, but that's how it works. We go in here and we want to create a mutation to create a new todo, passed in like that, want to use the create to do mutation that we just created and pass in the title, and then we’ll say that it is indeed done. We’re going to give this a true. You see I’m still henpecking but it smart autocorrects everything which is nice. And we’re going to get a to do object back because we defined this to do object because return value. We’ll get that.

Want the ID and then the title of what we get back. So create that, we’ll see its got the title “hello world” ID 6, if we go back to our query we had earlier. You can see that it saved just fine and it shows up at the bottom, that writing there. Now we’ve got the tools that we need to create queries that allow us to access our data and return it and filter it based on that we pass in.

We’ve got the tools that we need to create mutations which allows us to create, read, update, delete functions which allows us, pretty much the building block of our API right there and then we’ve got the type system which kind of smartly handles our Django models and passes them back to us in the exact format we want it only passes back the data that we actually query for.

There is another part to GraphQL which calls subscriptions - I put a bit of an asterix at the beginning because this isn’t fully implemented yet in graphene, there isn't like a plug and play implementation that you can just start playing with right away. I’m going to link the slide later on, there are some links to things you can look at if you want to create your own implementation but this would be the syntax.

You can subscribe to a todo updated event that would get fired, in this case we’d set it so it gets fired every time a user updates a todo model in the backend. It's going fire this event to us, what will happen is our client on the front end will connect to out backend view via a webhook and then that data will be passed via a long running webhook connection. That's not implemented in graphene yet but thats one of the things we’re really excited about because our app relies on a lot of real time stuff, so we are working on our own implementation that may possibly be open source at some point.

There's no plug in play support yet but there is a github that is linked here that has a lot of people attempts to implement this. If you are interested in that you can read along there, there's actually some interesting reading.

Other real time options, there are a couple other things out there, none of these are plug and play, they do require some set up. There is an official one at the top that does work with channel one. We’re using channels two, which we ran into a little while building the original API. we’re using this for the approach right here, we have a hybrid websocket connection where we are using raw websockets and the frontend rights it directly to GraphQL cache. If you are interested there are some links there so you can keep doing some reading.

Other things you might run into building the api file uploads - we want users to be able to upload a file for their profile picture, so that's super easy to do with a really sweet plugin definitely use that if you're looking for that functionality as well as there's some really cool utilities in the third party package called graphene django extras. If you want to port your serializer from django rest framework really quickly there's some cool stuff in here to do that. Handling authentication with decorator's is easier in here.

We’ve been running GraphQL in production for a couple months now and we’ll go through pros and cons a little bit and then we’ll  be done. One of the biggest pros for us is it dramatically improves our frontend developer experience. A lot of things we had to do manually on the front end - fetching the data, normalizing it, throwing it in the cache, making sure we're keeping the cache up to date, rendering it out to the user, are all more or less taken care of automatically in a declarative nature. We say we want this data and we want it displayed here and everything else is taken care of smartly behind the scene by GraphQL.

Flexible types have allowed us to speed up our development process so we’re not tied down by having to version apis, having to worry about backwards compatibility.  There's some cool tools in there that allow us to really quickly iterate on it and easily deprecate things. The documentation, this is less of a tools problem and more of just a people problem, we always have a hard time keeping our documentation and always have people asking each other “hey what about that thing that says this is the documentation” so because of the live playground that you get with GraphQL that's not really an issue for us anymore and we’re reducing our bandwidth usage because we’re only sending the data overt the wire that we actually need and not the whole data dump that you get with REST.

There are some cons with using it. One con is that Graphene isn’t quite as mature as other GraphQL on the backend, so Ruby and js have really solid mutations and GraphQL is super awesome and has a lot of good stuff going on but it’s still lagging just a little bit behind. A lot of opportunities to grow but thats one of the things we’ve run into a little bit.

Logging is different because you're using a single endpoint, so if you are using logging tools that are expecting a different endpoint for each rest endpoint that you are using you might have to look at how you are doing logging and make sure you’re looking a little bit deeper into those requests and know what's going on. And this is another big one, if you're using a lot of caching for production environment, REST is definitely better at caching at an http request level because GraphQL does this whole one endpoint thing where it only sends post requests its harder to take advantage of things like Etag and some of the built in caching mechanisms that you get with HTTP verbs when using them properly, so if you're really using that heavily you might take a hit on caching side of things, but what you lack in that you make up for in better application site caching so the client does a really smart cache enabled on the front end that kid shifts a lot of that onto the client.

Whats next? Graphene 3 is coming out soon, they've got a roadmap up on their github, a lot of cool stuff coming in, improvements on documentation, rewriting some of the code, great opportunity to get involved if you want to get involved in the GraphQL community and the django community in general because they are definitely looking for help here. We’re also working on some cool projects internally. We’re working on our own subscriptions implementation right now. It’s not in production, it's still demos right now but once we've got it running in production for a little bit there's some exciting open source possibilities that may come soon.