Dreamstate Coding: Functional Adventures in F# - Persisting Application State

Storing all actions

The idea is pretty simple, we will store all actions sent to the AppHolder MailboxProcessor and whenever we want to re-initiate the application state, we just load all the actions from storage and run them through the processor again and we should end up with the same application state.

This has been described by other people, for example Martin Fowler, if you have the time then head over there and read his article about event sourcing.

For this we will create a module called AppPersister. It will handle the actual storing and retrieving of the events from disk. So it will have side-effects and thus not be purely functional.... So lets just throw all those concepts out the door. But, this is a great opportunity to show some nice features of F# that will help write this kind of code a little more ... functional?
For serialization I chose to use the json format, and serializer is the JSON.NET library by Newtonsoft that can be added to your project with NuGet.

Some code for starters:

module AppPersister =
    open System
    open System.Reflection
    open System.IO
    open Newtonsoft.Json
    
    let private store = @".\store\"

Basically we just create a new module that will handle the persisting of actions... Here we also define that the store should be a subdirectory for the application.

    let private createActionFilename (action:obj) =
        let now = DateTime.UtcNow
        let hourPath = now.ToString("yyyy-MM-dd HH")
        let fullPath = Path.Combine(store, hourPath)
        let di = new DirectoryInfo(fullPath)
        di.Create()
        let t = action.GetType()
        let filename = now.ToString("yyyy-MM-dd hh_mm_ss_fffffff+") + now.Ticks.ToString() + "." + t.Name + ".action"
        let fullFilename = Path.Combine(fullPath, filename)
        fullFilename

The createActionFilename function does just that, it generates a new unique filename for the action by combining todays date and time, throwing in the total number of ticks to ensure that we always have a fresh value and lastly adding the type of action and the extension '.action'. Here we also create the directory if it does not exist already for the current date and hour (UTC), the DirectoryInfo.Create method is safe to run on an already existing directory so we do no other checking ourselves.

    let PersistAction (action:obj) =
        let fullFilename = createActionFilename action
        let json = JsonConvert.SerializeObject(action)
        File.WriteAllText(fullFilename,  json)
        ()

The PersisAction function handles the actual writing of the action to disk. We call createActionFilename to get the full filename and then use JsonConvert to serialize the action to a json string. Lastly we write the file with the nice File.WriteAllText method.

Now that we can write the actions, lets write some code to read them from disk.

    let GetAllActions () =
        let di = new DirectoryInfo(store)
        let actions =
            di.GetFiles("*.action", SearchOption.AllDirectories)
            |> Seq.map (fun (fi:FileInfo) -> File.ReadAllText(fi.FullName), fi.Name)
            |> Seq.map (fun x -> (getAction x))
            |> Seq.toArray
        actions

Here we use the DirectoryInfo.GetFiles built in method to find all files with the .action extension in any subdirectory in the store path. The result is an array of FileInfo objects that we pipe to a
Seq.map where we return a Tuple containing the file contents and filename and pipe that tuple into
another Seq.map where we call getAction for all elements. This function will be tasked with deserializing the json to the correct type
Lastly we pipe the contents to Seq.toArray to make the result concrete. To my understanding is that the Seq constructs work a little like the IEnumerable and do lazy evaluation if you do not actually list it.

    let private getAction (json:string, filename:string) =
        let split = filename.Split('.')
        let actionName = split.[1]
        let actionNameWithNamespace = "dreamstatecoding.core.Actions+" + actionName
        let t = Assembly.GetExecutingAssembly().GetType(actionNameWithNamespace)
        JsonConvert.DeserializeObject(json, t)

Lastly, the getAction function that is called from GetAllActions. Here we split the filename and pick out the part containing the name of the Action and then as the currently executing assembly to find the Type for that. Notice that we need to prefix the action name with the namespace name to make it work. In my solution I have all my actions in 1 module, so this works.

Rewriting AppHolder to work with persisted actions

Next step is to rewrite the AppHolder module from previous part to work with the AppPersister module.

So what we want to do here is to separate actual real time Actions from Replay actions from the persisting, mainly so that they do not get persisted again (duplicated).

So, lets put up a new discriminated union that defines what we want to do

    type Message =
        | Replay of obj
        | Action of obj

So, either we want to execute a Replay object or execute an Action object.

    let private Processor = Agent.Start(fun inbox ->
        let rec loop (s : AppliationState) =
            async {
                let! message = inbox.Receive()
                let (action:obj) = 
                    match message with
                    | Action a -> 
                        AppPersister.PersistAction a
                        a
                    | Replay a -> a

                let s' = s.HandleAction action
                state <- s'
                counter <- counter + 1
                return! loop s'
                }
        loop AppliationState.Default)

Now we rewrite our MailboxProcessor (Agent) to take a message instead of object directly. The first step is to pattern-match the received message to the type and get out the payload object.
If we are handling an Action we want to persist it as well, so lets put that function call here as well. Otherwise no change here.

    let HandleAction (action:obj) =
        Processor.Post (Action action)

The old HandleAction code called Processor.Post with the action directly. Now we must state that it is an Action, and not a Replay.

    let private HandleReplayAction (action:obj) =
        Processor.Post (Replay action)

The same goes for the Replay execution, we just add a new function that executes actions as Replays

    let InitiateFromStore () =
        AppPersister.GetAllActions()
        |> Array.map (fun x -> (HandleReplayAction x))
        |> ignore
        ()

Lastly, a function to initialize the store, we pipe all the actions loaded from persistence to the HandleReplayAction and then pipe the results from that to ignore. This means that we are not really interested in the results from this function call, in the end we just return unit from the InitiateFromStore function.
I put the call to this into my Program.cs file, it will load all the actions already executed from the store and execute them all as Replay actions.

Conclusions

In this part we have looked at how to persist the application state as a series of actions and then re-creating the application state when the application is restarted.
For applications handling lots and lots of actions, we may want to limit the number of actions needed to recreate the up to date state.. So in the next part we will look at snapshots. My plan is to upload the code to git after the snapshot part has been added.

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!

Dreamstate Coding

Pages

Wednesday, May 9, 2018

Functional Adventures in F# - Persisting Application State

Storing all actions

Rewriting AppHolder to work with persisted actions

Conclusions

No comments:

Post a Comment

Orbital velocity and altitude calculations in C#