Big Spatial Data Visualization using DeckGL

Big Data Visualization using DeckGL

deck.gl is a WebGL-powered framework for visual exploratory data analysis of large datasets.”

In this tutorial, we will learn how to visualize large geospatial datasets using deck.gl through an example. We will take uk accidents data sets and will visualize it using deck.gl.
In this tutorial, we have considered the UK accidents dataset from 2005 to 2104 which has around 1.5 Million points of occurred accidents for that period. This dataset have location coordinates that will be used in deck.gl for visualization.

Deck.gl is ReactJS friendly. It can handle efficient WebGL rendering under heavy data load.

To get full source code for this tutorial, click here.

To check the live demo for this tutorial, click here.

We will divide this tutorial in three parts:

  1. Preparing the UK accidents dataset.
  2. Setting ReactJS project.
  3. Visualizing data points using deck.gl.

1. Preparing the UK accidents dataset

We will use python to validate and clean up the UK accidents data. We will only validate location coordinates. We will remove invalid coordinates from dataset. The reason for validating location coordinates is because invalid coordinates may cause problem during data visualization in deck.gl.
The following code in python will validate the UK accidents location data and it will save the final validated data in JSON format which will be used in deck.gl for visuaization.
Python related code for data processing is in deckgl_app/data_processing_using_python path in github repo.

  1. We are using pandas to read UK accidents dataset.
  2. Then, at line number 13 and 14, we are converting the data type of longitude and latitude into number type.
  3. Then, at line number 17, we are using geopandas to convert the longitude-latitude columns into geodataframe. Using geopandasframe, we can validate invalid and empty coordinates or geometries.
  4. At line number 21 and 22, we are removing invalid and empty longitude-latitude coordinates from dataset. This makes dataset cleaner and ready to use.
  5. And then finally, we are storing the validated the longitude-latitude coordinates in JSON format.
  6. We will use this JSON file in deck.gl app for visualization. This file has around 1.5 Million points.

JSON file we created above looks like below sample:

2. Setting ReactJS project

DeckGL is react friendly. “deck.gl APIs are designed to reflect the reactive programming paradigm. Whether using Vanilla JS or the React interface, it can handle efficient WebGL rendering under heavy data load”. Because of that reason, we will set ReactJS project and install required dependencies.
Use the following commands to create ReactJS project and to install required dependencies:

Our default app structure looks like following:

3. Visualizing data points using deck.gl

Our final step is to visualize the point data we have created in first step of this tutorial using python. Our coordinate data for UK accidents is ready.
Now we will write deck.gl code in ReactJS app we created in step 2 of this tutorial to visualize the data points.
1. First, we will copy JSON file into reactjs app. We will copy this file under app->src->jsons directory with uk_accidents.json name.


2. Then, we will overwrite app.js file with our deckgl code that will visualize the data points on map.

Lets talks about deck.gl specific code that we wrote to visualize large data points.

  1. First, we are importing required dependencies like React, DeckGL, TileLayer, HexagonLayer etc.
  2. We are importing accident JSON file as module at line number 8 and giving alias to it as accidentss_jsons.
  3. We will use React module to create react component.
  4. We will use DeckGL module to render map on browser.
  5. We will use TileLayer module to create OpenStreetMap layer and will be displaying it as basemap.
  6. We will use HexagonLayer module to create 3D hexagon layer which will take JSON file as input and render it on map as 3D hex bars.

How the Visualization is happening:

We are using HexagonLayer module for visualization. This HexagonLayer layer takes accidents JSON file as input. Now we have to define radius value for hex bins. See line number 127, we are defining hex radius of 1000 meter. This means that hex bins of 1000 meter radius will be created on area of interests and their height will be based on number of data points(accidents JSON file) intersecting with each of hex bin. More the height of hex bin, more occurrences of accidents will be there.
This hexagon visualization will give clear indication at which location more accidents have happened based on hex bin height.

Our new project directory structure looks like below:

I hope this tutorial will create a good foundation for you. If you want tutorials on another GIS topic or you have any queries, please send an email at contact@spatial-dev.guru

To get full source code for this tutorial, click here.

To check the live demo for this tutorial, click here.

Leave a ReplyCancel reply