XTDB over HTTP
Without writing a single line of code, this tutorial will show you how to spin up an in-memory XTDB node on your development machine in less than 2 minutes. Then we’ll show you how to talk to it over HTTP from the command line.
| This tutorial will be skipping a large amount of information about XTDB, how it works, and much of its available functionality. For more information about the XTDB HTTP API, please take a look at the XTDB HTTP module documentation. Throughout this tutorial, you will see links to targeted extra reading and information, should you wish to read more. |
Part 1: Getting an XTDB node running on your local machine
This tutorial will install XTDB using the Clojure CLI tools. You can find in the other methods of installing XTDB in the XTDB installation docs.
If you do not have the Clojure CLI tools installed, installers are available from the Clojure Getting Started Guide. Don’t worry, you don’t need to know Clojure or the Clojure toolchain to run XTDB. Just copy the commands below.
(1) First, create a directory to house the settings for your new XTDB node by running the following in your terminal:
mkdir xtdb-cli && cd xtdb-cli(2) Next, create two new files within this directory named deps.edn and xtdb.edn with the following configuration code:
{:deps {org.clojure/clojure {:mvn/version "1.10.3"}
com.xtdb/xtdb-core {:mvn/version "1.24.3"}
com.xtdb/xtdb-http-server {:mvn/version "1.24.3"}}}{:xtdb.http-server/server {:port 3000}} (1)| 1 | This configuration tells XTDB to start an HTTP server on port 3000. |
(3) Last, and still from within xtdb-cli, run this command in your terminal:
clojure -M -m xtdb.main (1)| 1 | You will probably see a SLF4J: Failed to load class error. Just ignore it. |
Two minutes are up and your XTDB node is running. Nice work! To check that everything is working, head over to http://localhost:3000 where you should see the XTDB console.
Part 2: Talking to XTDB over HTTP
At the moment, the database is empty. Before we can start to play with some of the features that XTDB has to offer, we need to add some data. We will be using the REST API to send and receive JSON via curl.
For this tutorial, we will be working with facts about the current president of the United States. At the time of writing, the current president is Joe Biden. Lets add that to the database:
curl -X POST \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-d '{"tx-ops": [
["put", { (1)
"xt/id": "president", (2)
"name": "Joe", (3)
"last-name": "Biden" (4)
}]
]}' \
http://localhost:3000/_xtdb/submit-tx| 1 | put - The type of operation we are performing |
| 2 | The ID for the document we are adding |
| 3 | The name to be associated with president |
| 4 | The last name to be associated with the president |
We see that a transaction has been received by our locally running XTDB node. Now lets retrieve the entity:
curl -X GET \
-H "Accept: application/json" \
http://localhost:3000/_xtdb/entity\?eid=president (1)| 1 | GET the entity by the same ID we used above |
We see that the data we just put in the database is successfully showing up at the ID president.
Joe Biden hasn’t always been the president of the United States.
With XTDB, we can add data to the past (or the future) by passing a 'valid time' along with the document. Notice that we are using the same ID as before, something that wouldn’t work in a traditional database.
curl -X POST \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-d '{"tx-ops": [
["put", {
"xt/id": "president", (1)
"name": "Donald",
"last-name": "Trump"
}, "2017-01-20"] (2)
]}' \
http://localhost:3000/_xtdb/submit-tx| 1 | The same ID as our first put operation |
| 2 | The valid time for the document being added |
The above put operation places Donald Trump at the ID president from a valid date of 2017-01-20, his inauguration date.
Now lets retrieve the president entity again:
curl -X GET \
-H "Accept: application/json" \
http://localhost:3000/_xtdb/entity\?eid=presidentWe get the same result - Joe Biden!
The GET request of /entity returns the data that is currently valid for the ID president.
This is the first example of the power of a temporal database. We are able to ingest data to provide a better history of data should we want to look back.
To retrieve the Donald Trump document we just added, we can pass a valid time with our GET request:
curl -X GET \
-H "Accept: application/json" \
http://localhost:3000/_xtdb/entity\?eid=president\&valid-time=2020-10-10 (1)| 1 | We pass a valid time of 2020-10-10 |
The above operation allows us to find out who the president was on the date passed. XTDB will return Donald Trump for any date between 2017-01-20 and the valid time on the Joe Biden document.
We can see all of our transactions by passing some more arguments to the same /entity endpoint:
curl -X GET \
-H "Accept: application/json" \
http://localhost:3000/_xtdb/entity\?eid=president\&history=true\&sortOrder=desc\&withDocs=true (1)| 1 | history=true returns all historical documents for this entity |
If you look closely at the Joe Biden document, the validTime is equal to today’s date. This is because we did not explicitly provide a 'valid time' with our initial put. By default, XTDB assumes validTime is equal to the current date-time unless otherwise specified.
Let’s correct the entry for Joe Biden by passing the correct 'valid time':
curl -X POST \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-d '{"tx-ops": [
["put", {
"xt/id": "president",
"name": "Joe",
"last-name": "Biden"
}, "2021-01-20"] (1)
]}' \
http://localhost:3000/_xtdb/submit-tx| 1 | Again, "2021-01-20" is our explicit 'valid time' parameter. |
We now have a historically accurate dataset for the last two presidents of the United States that we can query over a temporal plane! You can run the history query again to validate our correction.
Using XTDB, we can do powerful queries over the temporal plane and retrieve data as if we had travelled back in time.[1] XTDB achieves this by maintaining an immutable (write-only) transaction log. This makes XTDB a powerful asset where auditing is important or when looking back at historical data is valuable. Better still, XTDB can build on top of many different DB solutions allowing you to retain the infrastructure you already know and love!
In this tutorial, we used 3 operations: storing documents with submit-tx, retrieving documents with entity, and retrieving document histories with history=true. The complete set of 18 REST operations, including RESTful Datalog queries, are explained in the XTDB REST API documentation.
XTDB also supports a number of other protocols and features:
Now that you’ve dipped your toes in the temporal data waters, we encourage you to experiment with the more advanced features of XTDB!