Serialize EF Core InMemory Database to JSON

c# ef-core-2.0 entity-framework entity-framework-core json

Question

I'm looking forward to extract delta changes tons of entities in DbContext, and delegate actual DB commit to background process, say Azure web job.

Attempted this, but couldn't serialize.

        var deltaJson = "";
        try
        {
            var modifiedEntries = _ctx.ChangeTracker
                .Entries()
                .Select(x => x.Entity)
                .ToList();
            deltaJson = JsonConvert.SerializeObject(modifiedEntries, Formatting.Indented);
        }

My next hope is to use in memory Database and possibly if we could serialize entire object graph of DbContext.

Is it feasible? Your expert advice & any pointers in this direction would be very helpful.

EDIT: My own version:

public class DeltaTracking
{
    public static List<T> Build<T>(ICollection<T> db, ICollection<T> req) where T : IR
    {
        return Build(db, req, new DT<T>());
    }

    public static List<T> Build<T>(ICollection<T> db, ICollection<T> req, IEqualityComparer<T> comp) where T : IStateTracking
    {
        db = db ?? new List<T>();
        req = req ?? new List<T>();

        List<T> added = req.Except(db, comp).ToList();
        foreach (T a in added)
        {
            a.State = TrackingState.Added.ToString();
        }

        List<T> removed = db.Except(req, comp).ToList();
        foreach (T r in removed)
        {
            r.State = TrackingState.Deleted.ToString();
        }

        List<T> unchanged = db.Intersect(req, comp).ToList();
        foreach (T u in unchanged)
        {
            u.State = TrackingState.Unchanged.ToString();
        }
        List<T> resp = added.Union(removed, comp).Union(unchanged, comp).ToList();
        return resp;
    }
}
1
0
8/22/2018 6:46:39 AM

Accepted Answer

You can serialize entities from the change tracker, however given this sounds like you want to serialize and offload a large number of changes you will probably need to package up the changes in smaller pages, transfer them across the wire, track and compose the set of changes on the other side and ensuring they are reassembled in order since the changeset will span related entities.

const int pageSize = 1000;
var count = context.ChangeTracker.Entries()
    .Count(x => x.State == EntityState.Modified || x.State == EntityState.Added || x.State == EntityState.Deleted);

var pages = (int) Math.Ceiling((double) count / PageSize);
int loopCounter = 0;
while (loopCounter < pages)
{
   var changes = context.ChangeTracker.Entries()
      .Where(x => x.State == EntityState.Modified || x.State == EntityState.Added || x.State == EntityState.Deleted)
      .Select(x => new { Type = x.Entity.GetType(), State = x.State.ToString(), Original = x.OriginalValues.ToObject(), Updated = x.CurrentValues.ToObject() })
      .Skip(loopCounter * pageSize)
      .Take(pageSize);

   var data = new
   {
      PageNumber = loopCounter,
      Changes = changes,
   };

   string changeData = JsonConvert.SerializeObject(data, Formatting.Indented);
   // Fire across to destination to be processed. Be sure to send the total # of pages because the server cannot start processing these unless they are in order.

.Changes would contain the Type, state (what action to take, update, add, delete) and the original and updated entity values applicable.

If this is only handling batch updates, then exclude the Add/Delete items and let those flow through.

On the receiver side I don't believe it will be as simple as taking the changed entities and attaching them to a DbContext to be persisted for updates. These would likely need to be loaded by the destination DbContext and updated based on the fields in the attached entities. There would be a bit of work to extract things like PKs for Delete operations. Adds should be pretty straight forward.

The other real test would be concurrency between the time that the changes are made, and when they will be run by the background service. You should consider checking versioning for any updates. This likely isn't a complete solution, but hopefully it gives some ideas or sparks some discussion.

1
8/23/2018 12:18:23 AM


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow