Assembly 'System.Memory' not found when using Entity.FrameworkCore.Sqlite in Xamarin project

I am trying to get a project based on Xamarin with a .Net standard project and entity framework core to work, but I cannot resolve the following error:

Exception while loading assemblies: System.IO.FileNotFoundException: Could not load assembly 'System.Memory, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. Perhaps it doesn't exist in the Mono for Android profile? File name: 'System.Memory.dll' at Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.Resolve(AssemblyNameReference reference, ReaderParameters parameters) at Xamarin.Android.Tasks.ResolveAssemblies.AddAssemblyReferences(DirectoryAssemblyResolver resolver, ICollection`1 assemblies, AssemblyDefinition assembly, Boolean topLevel) at Xamarin.Android.Tasks.ResolveAssemblies.AddAssemblyReferences(DirectoryAssemblyResolver resolver, ICollection`1 assemblies, AssemblyDefinition assembly, Boolean topLevel) at Xamarin.Android.Tasks.ResolveAssemblies.AddAssemblyReferences(DirectoryAssemblyResolver resolver, ICollection`1 assemblies, AssemblyDefinition assembly, Boolean topLevel) at Xamarin.Android.Tasks.ResolveAssemblies.Execute(DirectoryAssemblyResolver resolver)

For the reference management I am using the nuget package manager in VS2017 and the project is setup to use the packages.config as suggested here.

The nuget package System.Memory is installed for the Android project, so I'm a bit clueless what could cause the error. Only the version is different, installed is version 4.5.0 and according to the error message it's checking for 4.0.1.0, yet this version does not exist for System.Memory.

Anyone can relate to this error or provide any hints / help?