Mastering Data Persistence in Roblox
Have you ever been curious how to integrate distributed Roblox servers with their distributed Datastore?
I'm going to explain my process for integrating with the Roblox database assuming a beginner perspective. Note that I will assume that the reader has some level of programming knowledge, this is not a programming tutorial.
The Setup
- First, you must explicitly enable HTTP Requests in your Roblox Studio settings.
- Then, get the ProfileStore module from Loleris which can be found in the toolbox or on github.
What is ProfileStore? It's an open source module used to integrate with Roblox's database. It handles complex edge cases like session locking and race conditions.
To add ProfileStore to your project, go to the Toolbox and search “ProfileStore” under the “Models” category. It should be the first result by Loleris.
Understanding the Steps of the Process
What do we want to accomplish?
- Define our data model: We want a typed definition of our data model. This is useful because it will serve as documentation for all of the data we’re storing for our game. It will be easy for you to reference if you ever forget something. Defining a type also gives us the benefit of autocomplete, and if you have --!strict mode on your script, it will give you type checking as well.
- Load player data when the player joins the game: When a player joins our game, we need to load their previously saved data.
- Save player data while player is playing: While the player is in our game, we need to save their data so that it persists across play sessions.
- Save when player leaves or the server shuts down: We need to make sure to save the player data when they leave the game. We also need to make sure to handle the scenario where the Roblox server gets terminated for some reason.
Here is each step broken down:
Define our Data Model
We want a typed definition of our model. How do we accomplish this? Well, ProfileStore makes use of something called Profiles. A Profile object is what we use to read from and write data to the database. Luckily, the ProfileStore module comes with types, and we can build off of them to define our data model.
In the ProfileStore script, if you do control + f (command + f on mac), it will bring up a text input we can use to search for any terms in the script. Type the following: export type Profile and press Enter. It should take you to the definition of Profile, which looks like this:
1export type Profile<T> = {
2 Data: T & JSONAcceptable,
3 LastSavedData: T & JSONAcceptable,
4 ...
5 ...
Go ahead and remove JSONAcceptable as a type from Data and LastSavedData. Why? When it has that type, Luau’s type checker gets confused, so let’s remove it. After you remove it, the type definition should look like this:
1export type Profile<T> = {
2 Data: T,
3 LastSavedData: T,
4 ...
5 ...
Great! Now we can define our data model. Personally, I did this in a file called PlayerDataTemplate. I named it this because one cool feature of ProfileStore is that Player data is initialized to the default values you specify in your data model. This applies to players who are joining your game for the very first time.. Other names such as DataModel, PlayerData, etc would also work.
Make sure to add --!strict to the top of your file so that type checking is applied.
Here’s a starting version of PlayerDataTemplate.lua
1--!strict
2local ServerStorage = game:GetService("ServerStorage")
3local ReplicatedStorage = game:GetService("ReplicatedStorage")
4
5-- Assumes a Folder named ModuleScripts is a child of ServerStorage,
6-- a Folder named Dependencies is a child of ModuleScripts, and
7-- a ModuleScript named ProfileStore is a child of Dependencies.
8local ModuleScripts = ServerStorage:WaitForChild("ModuleScripts")
9local Dependencies = ModuleScripts:WaitForChild("Dependencies")
10local ProfileStore = require(Dependencies:WaitForChild("ProfileStore"))
11
12-- Defining our own type of profile by passing in our PlayerDataTemplate type
13export type Profile = ProfileStore.Profile<PlayerDataTemplate>
14-- Type definition for PlayerDataTemplate, we haven't defined anything yet
15export type PlayerDataTemplate = {}
16-- Default values of PlayerDataTemplate. Players that are brand new to your game
17-- will start out with this data
18local PlayerDataTemplate: PlayerDataTemplate = {}
19
20return PlayerDataTemplate
How will we define our model? Well, it’s important to know that data within the Roblox database exist as Tables. Therefore, we will define our model as a table. In this example, I’m going to show how someone might store XP, Level, and Titles. Before we do that, I want to quickly discuss limitations.
Limitations of what can be stored
First, we should know that the only types of values we can store are table, number, string, buffer, boolean, and nil.
Finally, all of the tables we store must have either all string indices or all number indices. This means all keys of a given table must be of the same type.
Model Our Data
XP and Level: This is the easiest one. XP and Level are both going to be represented as a number.
Our default template and type definition now looks like this:
1export type PlayerDataTemplate = {
2 XP: number,
3 Level: number,
4}
5
6-- Default values of PlayerDataTemplate. Players that are brand new to your game
7-- will start out with this data
8local PlayerDataTemplate: PlayerDataTemplate = {
9 XP = 0,
10 Level = 1,
11}
Titles: For Titles, we’re going to store two pieces of information. In a real game you may want more than this, but I’m sticking with two for the sake of simplicity.
The Title type will store the name of the title and the time the player earned the title. Storing the time the player earned a title could be useful for something such as giving founding players a reward.
Let’s translate that into a type.
1export type Title = {
2 Name: string,
3 -- Date the title was earned
4 EarnedAt: number,
5}
Now our definitions look like this. For your convenience, I am pasting the full script below:
1--!strict
2local ServerStorage = game:GetService("ServerStorage")
3local ReplicatedStorage = game:GetService("ReplicatedStorage")
4
5-- Assumes a Folder named ModuleScripts is a child of ServerStorage,
6-- a Folder named Dependencies is a child of ModuleScripts, and
7-- a ModuleScript names ProfileStore is a child of Dependencies.
8local ModuleScripts = ServerStorage:WaitForChild("ModuleScripts")
9local Dependencies = ModuleScripts:WaitForChild("Dependencies")
10local ProfileStore = require(Dependencies:WaitForChild("ProfileStore"))
11
12-- Defining our own type of profile by passing in our PlayerDataTemplate type
13export type Profile = ProfileStore.Profile<PlayerDataTemplate>
14
15export type Title = {
16 Name: string,
17 -- Date the title was earned
18 EarnedAt: number,
19}
20
21-- Type definition for PlayerDataTemplate
22export type PlayerDataTemplate = {
23 XP: number,
24 Level: number,
25 OwnedTitles: { Title },
26}
27
28-- Default values of PlayerDataTemplate. Players that are brand new to your game
29-- will start out with this data
30local PlayerDataTemplate: PlayerDataTemplate = {
31 XP = 0,
32 Level = 1,
33 OwnedTitles = {
34 {Name = "Noob", EarnedAt = tick()},
35 {Name = "Weakling", EarnedAt = tick()}
36 }
37}
38
39return PlayerDataTemplate
Now we have a basic definition for our data model. Let’s continue on to see how we can use this definition to load player data.
Loading Player Data When Player Joins
Now we want to load a player’s data when a player joins.
To do this we should understand some things:
We’re doing this on the Server, so this code should run by a script parented to ServerScriptService. There’s an event called PlayerAdded that we can use to perform some task when the player joins the game. I’m going to give the starting script with comments. This script should be a Script parented to ServerScriptService.
1--!strict
2
3-- Services
4local ServerStorage = game:GetService("ServerStorage")
5local ReplicatedStorage = game:GetService("ReplicatedStorage")
6local Players = game:GetService("Players")
7
8-- ModuleScipts
9local ModuleScripts = ServerStorage:WaitForChild("ModuleScripts")
10local Dependencies = ModuleScripts:WaitForChild("Dependencies")
11local ProfileStore = require(Dependencies:WaitForChild("ProfileStore"))
12local PlayerDataTemplate = require(ModuleScripts:WaitForChild("PlayerDataTemplate"))
13
14-- Name the ProfileStore appropriately, I chose "PlayerData" here
15local PlayerProfileStore = ProfileStore.New("PlayerData", PlayerDataTemplate)
16
17-- This function will be executed when a player joins our game
18local function OnPlayerAdded(player: Player)
19 -- Use a unique key to store the player data. Any string concatenated with the player's UserId should work.
20 local profileKey = "Test_" .. tostring(player.UserId)
21 -- Fetch the profile from the DB with session locking which prevents issues like item dupes.
22 local profile = PlayerProfileStore:StartSessionAsync(profileKey,
23 -- Only use Steal = True for debugging purposes. Setting Steal to true would disable item dupe protection
24 {Steal = false, Cancel = function()
25 -- ProfileStore will periodically use this function to check if the session needs to be ended.
26 -- Useful for some edge cases.
27 return player.Parent ~= Players
28 end}
29 )
30 if not profile then
31 -- If the profile doesn't exist, it's likely because Roblox itself is experiencing issues.
32 -- We don't know what to do with the player if their data doesn't load, so let's kick them.
33 player:Kick("Failed to load your data. Please try rejoining after a short wait.")
34 return
35 end
36
37 profile:AddUserId(player.UserId) -- This is for GDPR Compliance
38 profile:Reconcile() -- The reconcile function applies our default template values to new profiles.
39
40 print("Player level:", profile.Data.Level)
41 print("Player XP:", profile.Data.XP)
42 for _, title in pairs(profile.Data.OwnedTitles) do
43 print("Player owns title:", title.Name)
44 end
45end
46
47-- Load profiles for players that got in the game before this server script finished running
48local function LoadExistingPlayers()
49 for _, player in pairs(game.Players:GetPlayers()) do
50 task.spawn(OnPlayerAdded, player)
51 end
52end
53
54LoadExistingPlayers()
55game.Players.PlayerAdded:Connect(OnPlayerAdded) -- Connect the function to the player added event
If you did everything correctly, you can now test the experience. The output should look like this when you run the server:
04:48:52.401 [ProfileStore]: Roblox API services available - data will be saved - Server - ProfileStore:2087
04:48:53.477 Player level: 1 - Server - TeachMain:40
04:48:53.477 Player XP: 0 - Server - TeachMain:41
04:48:53.477 Player owns title: Noob - Server - TeachMain:43
04:48:53.477 Player owns title: Weakling - Server - TeachMain:43
Saving Player Data
Now we can successfully load player data. How do we go about saving player data though?
First, we will need a way to reference a previously created profile. To do this, let’s create a Table which will be used as a Map from Player.UserId to the player’s Profile.
Initialize this table outside the scope of the OnPlayerAdded function:
1local Profiles: {[number]: PlayerDataTemplate.Profile} = {}
Now we need to update Profiles with the Player.UserId and Profile. To do this, when a profile is successfully created, we do this;
1Profiles[player.UserId] = profile
Tying it all together in one script:
1--!strict
2
3-- Services
4local ServerStorage = game:GetService("ServerStorage")
5local ReplicatedStorage = game:GetService("ReplicatedStorage")
6local Players = game:GetService("Players")
7
8-- ModuleScipts
9local ModuleScripts = ServerStorage:WaitForChild("ModuleScripts")
10local Dependencies = ModuleScripts:WaitForChild("Dependencies")
11local ProfileStore = require(Dependencies:WaitForChild("ProfileStore"))
12local PlayerDataTemplate = require(ModuleScripts:WaitForChild("PlayerDataTemplate"))
13
14-- Name the ProfileStore appropriately, I chose "PlayerData" here
15local PlayerProfileStore = ProfileStore.New("PlayerData", PlayerDataTemplate)
16local Profiles: {[number]: PlayerDataTemplate.Profile} = {}
17
18-- This function will be executed when a player joins our game
19local function OnPlayerAdded(player: Player)
20 -- Use a unique key to store the player data. Any string concatenated with the player's UserId should work.
21 local profileKey = "Test_" .. tostring(player.UserId)
22 -- Fetch the profile from the DB with session locking which prevents issues like item dupes.
23 local profile = PlayerProfileStore:StartSessionAsync(profileKey,
24 -- Only use Steal = True for debugging purposes. Setting Steal to true would disable item dupe protection
25 {Steal = false, Cancel = function()
26 -- ProfileStore will periodically use this function to check if the session needs to be ended.
27 -- Useful for some edge cases.
28 return player.Parent ~= Players
29 end}
30 )
31 if not profile then
32 -- If the profile doesn't exist, it's likely because Roblox itself is experiencing issues.
33 -- We don't know what to do with the player if their data doesn't load, so let's kick them.
34 player:Kick("Failed to load your data. Please try rejoining after a short wait.")
35 return
36 end
37
38 Profiles[player.UserId] = profile
39 profile:AddUserId(player.UserId) -- This is for GDPR Compliance
40 profile:Reconcile() -- The reconcile function applies our default template values to new profiles.
41
42 print("Player level:", profile.Data.Level)
43 print("Player XP:", profile.Data.XP)
44 for _, title in pairs(profile.Data.OwnedTitles) do
45 print("Player owns title:", title.Name)
46 end
47end
48
49-- Load profiles for players that got in the game before this server script finished running
50local function LoadExistingPlayers()
51 for _, player in pairs(game.Players:GetPlayers()) do
52 task.spawn(OnPlayerAdded, player)
53 end
54end
55
56LoadExistingPlayers()
57game.Players.PlayerAdded:Connect(OnPlayerAdded) -- Connect the function to the player added event
Now we want to actually save the data. How do we do this? Well, we modify profile.Data directly. ProfileStore will periodically sync the data saved in memory inside of profile.Data to the database. This autosaving feature decreases load on our servers and helps us avoid the rate limits for DataStores.
There’s also another thing we need to do. We need to tell ProfileStore to save the current profile.Data to the database when the profile’s session is ending.
Another issue we have to deal with is memory leaks. Our current code adds an entry to Profiles every time a player joins, but it never removes any entries. This could be an issue if our game is getting lots of visits.
Finally, we need to account for the scenario that the player leaves the game before the profile finished loading.
To do this, we will add the following code:
1if player.Parent ~= Players then
2 -- The player left the game before we finished loading the profile
3 profile:EndSession()
4end
5
6profile.OnSessionEnd:Connect(function()
7 -- Free up space in Profiles. Without this, we could have a memory leak.
8 Profiles[player.UserId] = nil
9 player:Kick(`Profile session end - Please rejoin`)
10end)
Putting it all together:
1--!strict
2
3-- Services
4local ServerStorage = game:GetService("ServerStorage")
5local ReplicatedStorage = game:GetService("ReplicatedStorage")
6local Players = game:GetService("Players")
7
8-- ModuleScipts
9local ModuleScripts = ServerStorage:WaitForChild("ModuleScripts")
10local Dependencies = ModuleScripts:WaitForChild("Dependencies")
11local ProfileStore = require(Dependencies:WaitForChild("ProfileStore"))
12local PlayerDataTemplate = require(ModuleScripts:WaitForChild("PlayerDataTemplate"))
13
14-- Name the ProfileStore appropriately, I chose "PlayerData" here
15local PlayerProfileStore = ProfileStore.New("PlayerData", PlayerDataTemplate)
16local Profiles: {[number]: PlayerDataTemplate.Profile} = {}
17
18-- This function will be executed when a player joins our game
19local function OnPlayerAdded(player: Player)
20 -- Use a unique key to store the player data. Any string concatenated with the player's UserId should work.
21 local profileKey = "Test_" .. tostring(player.UserId)
22 -- Fetch the profile from the DB with session locking which prevents issues like item dupes.
23 local profile = PlayerProfileStore:StartSessionAsync(profileKey,
24 -- Only use Steal = True for debugging purposes. Setting Steal to true would disable item dupe protection
25 {Steal = false, Cancel = function()
26 -- ProfileStore will periodically use this function to check if the session needs to be ended.
27 -- Useful for some edge cases.
28 return player.Parent ~= Players
29 end}
30 )
31 if not profile then
32 -- If the profile doesn't exist, it's likely because Roblox itself is experiencing issues.
33 -- We don't know what to do with the player if their data doesn't load, so let's kick them.
34 player:Kick("Failed to load your data. Please try rejoining after a short wait.")
35 return
36 end
37
38 if player.Parent ~= Players then
39 -- The player left the game before we finished loading the profile
40 profile:EndSession()
41 end
42
43 profile.OnSessionEnd:Connect(function()
44 -- Free up space in Profiles. Without this, we could have a memory leak.
45 Profiles[player.UserId] = nil
46 player:Kick(`Profile session end - Please rejoin`)
47 end)
48
49 Profiles[player.UserId] = profile
50 profile:AddUserId(player.UserId) -- This is for GDPR Compliance
51 profile:Reconcile() -- The reconcile function applies our default template values to new profiles.
52
53 print("Player level:", profile.Data.Level)
54 print("Player XP:", profile.Data.XP)
55 for _, title in pairs(profile.Data.OwnedTitles) do
56 print("Player owns title:", title.Name)
57 end
58end
59
60-- Load profiles for players that got in the game before this server script finished running
61local function LoadExistingPlayers()
62 for _, player in pairs(game.Players:GetPlayers()) do
63 task.spawn(OnPlayerAdded, player)
64 end
65end
66
67LoadExistingPlayers()
68game.Players.PlayerAdded:Connect(OnPlayerAdded) -- Connect the function to the player added event
Great! Now let’s test that this is working. Let’s add a simple function that gives the user 1 XP for each second they’re in the game. That code looks like this, add it after the call to profile:Reconcile():
1task.spawn(function()
2 while true do
3 task.wait(1)
4 profile.Data.XP += 1
5 end
6end)
Now you can start your game. Let it run for a few seconds, and then leave the game. Now, in order to test that the data saved properly, let’s rejoin the game so we can see the newly printed out value of XP. Depending on how long you let the XP counter run, you will see a different result for the value of XP. Here’s what the output looked like to me:
05:35:11.394 [ProfileStore]: Roblox API services available - data will be saved - Server - ProfileStore:2087
05:35:12.427 Player level: 1 - Server - TeachMain:54
05:35:12.427 Player XP: 20 - Server - TeachMain:55
05:35:12.427 Player owns title: Noob - Server - TeachMain:57
05:35:12.427 Player owns title: Weakling - Server - TeachMain:57
Now that we have the ability to save our data to the data store, there’s one final thing we need to do, and that’s handling when the player leaves or the server shuts down.
Handle Player Leaving or Server Shutdown
How will we handle this? Let’s break it down for each case:
- Player leaving: We can hook an event up to the game.Players.PlayerRemoving signal to perform some task when the player leaves.
- Server shutdown: ProfileStore handles this for us! Experienced devs might notice I’m leaving out BindToClose. This is because ProfileStore already handles BindToClose, so we don’t need to. This means the scenario where the server shuts down is already handled for us and we don’t need any custom logic. Now let’s look at how we can save player data when they leave:
Save Player Data When They Leave
Let’s create an OnPlayerRemoving function. In it, we will simply call profile:EndSession. This will trigger one final save to the database before ending the session.
1local function OnPlayerRemoving(player: Player)
2 local profile = Profiles[player.UserId]
3 if profile then
4 profile:EndSession()
5 end
6end
7
8game.Players.PlayerRemoving:Connect(OnPlayerRemoving) -- Connect the function to the player removing event
Final Scripts
1-- PlayerDataTemplate (ModuleScript)
2--!strict
3local ServerStorage = game:GetService("ServerStorage")
4local ReplicatedStorage = game:GetService("ReplicatedStorage")
5
6-- Assumes a Folder named ModuleScripts is a child of ServerStorage,
7-- a Folder named Dependencies is a child of ModuleScripts, and
8-- a ModuleScript names ProfileStore is a child of Dependencies.
9local moduleScripts = ServerStorage:WaitForChild("ModuleScripts")
10local Dependencies = moduleScripts:WaitForChild("Dependencies")
11local ProfileStore = require(Dependencies:WaitForChild("ProfileStore"))
12
13-- Defining our own type of profile by passing in our PlayerDataTemplate type
14export type Profile = ProfileStore.Profile<PlayerDataTemplate>
15
16export type Title = {
17 Name: string,
18 -- Date the title was earned
19 EarnedAt: number,
20}
21
22-- Type definition for PlayerDataTemplate
23export type PlayerDataTemplate = {
24 XP: number,
25 Level: number,
26 OwnedTitles: { Title },
27}
28
29-- Default values of PlayerDataTemplate. Players that are brand new to your game
30-- will start out with this data
31local PlayerDataTemplate: PlayerDataTemplate = {
32 XP = 0,
33 Level = 1,
34 OwnedTitles = {
35 {Name = "Noob", EarnedAt = tick()},
36 {Name = "Weakling", EarnedAt = tick()}
37 }
38}
39
40return PlayerDataTemplate
1-- Main server script
2--!strict
3
4-- Services
5local ServerStorage = game:GetService("ServerStorage")
6local ReplicatedStorage = game:GetService("ReplicatedStorage")
7local Players = game:GetService("Players")
8
9-- ModuleScipts
10local ModuleScripts = ServerStorage:WaitForChild("ModuleScripts")
11local Dependencies = ModuleScripts:WaitForChild("Dependencies")
12local ProfileStore = require(Dependencies:WaitForChild("ProfileStore"))
13local PlayerDataTemplate = require(ModuleScripts:WaitForChild("PlayerDataTemplate"))
14
15-- Name the ProfileStore appropriately, I chose "PlayerData" here
16local PlayerProfileStore = ProfileStore.New("PlayerData", PlayerDataTemplate)
17local Profiles: {[number]: PlayerDataTemplate.Profile} = {}
18
19-- This function will be executed when a player joins our game
20local function OnPlayerAdded(player: Player)
21 -- Use a unique key to store the player data. Any string concatenated with the player's UserId should work.
22 local profileKey = "Test_" .. tostring(player.UserId)
23 -- Fetch the profile from the DB with session locking which prevents issues like item dupes.
24 local profile = PlayerProfileStore:StartSessionAsync(profileKey,
25 -- Only use Steal = True for debugging purposes. Setting Steal to true would disable item dupe protection
26 {Steal = false, Cancel = function()
27 -- ProfileStore will periodically use this function to check if the session needs to be ended.
28 -- Useful for some edge cases.
29 return player.Parent ~= Players
30 end}
31 )
32 if not profile then
33 -- If the profile doesn't exist, it's likely because Roblox itself is experiencing issues.
34 -- We don't know what to do with the player if their data doesn't load, so let's kick them.
35 player:Kick("Failed to load your data. Please try rejoining after a short wait.")
36 return
37 end
38
39 if player.Parent ~= Players then
40 -- The player left the game before we finished loading the profile
41 profile:EndSession()
42 end
43
44 profile.OnSessionEnd:Connect(function()
45 -- Free up space in Profiles. Without this, we could have a memory leak.
46 Profiles[player.UserId] = nil
47 player:Kick(`Profile session end - Please rejoin`)
48 end)
49
50 Profiles[player.UserId] = profile
51 profile:AddUserId(player.UserId) -- This is for GDPR Compliance
52 profile:Reconcile() -- The reconcile function applies our default template values to new profiles.
53
54 print("Player level:", profile.Data.Level)
55 print("Player XP:", profile.Data.XP)
56 for _, title in pairs(profile.Data.OwnedTitles) do
57 print("Player owns title:", title.Name)
58 end
59
60 task.spawn(function()
61 while true do
62 task.wait(1)
63 profile.Data.XP += 1
64 end
65 end)
66end
67
68local function OnPlayerRemoving(player: Player)
69 local profile = Profiles[player.UserId]
70 if profile then
71 profile:EndSession()
72 end
73end
74
75-- Load profiles for players that got in the game before this server script finished running
76local function LoadExistingPlayers()
77 for _, player in pairs(game.Players:GetPlayers()) do
78 task.spawn(OnPlayerAdded, player)
79 end
80end
81
82LoadExistingPlayers()
83game.Players.PlayerAdded:Connect(OnPlayerAdded) -- Connect the function to the player added event
84game.Players.PlayerRemoving:Connect(OnPlayerRemoving) -- Connect the function to the player removing event
Conclusion
That's how you can integrate with the Roblox database in a type-safe way. Hope you enjoyed the tutorial!