首页 > 代码库 > leveldb.net对象读写封装

leveldb.net对象读写封装

2024-07-20 23:11:18   211人阅读

        leveldb是一个非常高效的可嵌入式K-V数据库,在.NET下有着基于win实现的包装leveldb.net;不过leveldb.net只提供了基于byte[]和string的处理,这显然会对使用的时候带来不方便,毕竟在编写应用的时候都是希望通过对象的方式来存储,如我们常见的redis,mongodb和memcached等等都提供对象方式的读写.以下主要讲解leveldb.net基础上封装一层序列化功能方便使用.

制定对象化的访问接口

为了不修改leveldb.net的代码,所以选择在他基础过行封装,为了清楚需要些什么简单地定义了一个规则

 public interface IDBManager    {        IFormater Formater { get; set; }        void Set(string key, object data);        object Get(string key, Type type);        T Get<T>(string key);        void Open();        LevelDB.DB DataBase        {            get;        }           }

代码非常简单主要封装了GET,SET,实际上还有DELETE操作,这里偷懒就没做了:),为了提供灵活序列化规则所以在这个管理接口上还提供了一个Formater属性.下面是这相接口的详细实现:

 

public class LevelDBManager : IDBManager    {        public LevelDBManager()        {                    }        private LevelDB.DB mDataBase;        public string Path { get; set; }        public IFormater Formater        {            get;            set;        }        public void Open()        {            mDataBase = new LevelDB.DB(Path, new Options() { CreateIfMissing = true });                  }        public void Set(string key, object data)        {            FormaterBuffer buffer = Formater.Pop();            try            {                int count = Formater.Serialize(data, buffer, 0);                mDataBase.Put(Encoding.UTF8.GetBytes(key), buffer.Array, 0, count);            }            finally            {                Formater.Push(buffer);            }        }        public object Get(string key, Type type)        {            FormaterBuffer buffer = Formater.Pop();            long count;            object result = null;                       try            {                count = mDataBase.Get(Encoding.UTF8.GetBytes(key), buffer.Array);                if (count > 0)                {                    result = Formater.Deserialize(type, buffer, 0, (int)count);                }                return result;            }            finally            {                Formater.Push(buffer);            }                  }        public T Get<T>(string key)        {            return (T)Get(key, typeof(T));        }        public DB DataBase        {            get { return mDataBase; }        }    }

相信以上那些简知的代码也比较好理解,所以就不详细说明了.

可扩展的序列化规则

由于在使用上的需要,都习惯用些不同序列化方式来进行对象序列化,这个封装为了实现一个比较高的灵活度,所以对象序列化过程也制定了一个接口进行隔离.主要为了满足不同人的胃口.

 

   public interface IFormater    {        FormaterBuffer Pop();        void Push(FormaterBuffer data);        int Serialize(object data, FormaterBuffer buffer, int offset);        object Deserialize(Type type, FormaterBuffer buffer, int offset, int count);    }

比较简单定义了序列化和反序列化的方法,不过为了一些性能上的考虑增加了buffer的复用功能,这个设计紧紧用作需要追求这方面性能要求而准备.下面看一下json和protobuf的实现是怎样的:

 public abstract class FormaterBase:IFormater    {         private Stack<FormaterBuffer> mBufferPool = new Stack<FormaterBuffer>();        const int BUFFER_SIZE = 1024 * 1024 * 1;        public FormaterBase()        {            for (int i = 0; i < 20; i++)            {                mBufferPool.Push(new FormaterBuffer(BUFFER_SIZE));            }        }        public FormaterBuffer Pop()        {            lock (mBufferPool)            {                if(mBufferPool.Count>0)                    return mBufferPool.Pop();                return new FormaterBuffer(BUFFER_SIZE);            }        }        public void Push(FormaterBuffer data)        {            lock (mBufferPool)            {                mBufferPool.Push(data);            }        }               public abstract int Serialize(object data, FormaterBuffer buffer, int offset);               public abstract object Deserialize(Type type, FormaterBuffer buffer, int offset, int count);            }
  • json
    public class JsnoFormater:FormaterBase    {               public int Serialize(object data, byte[] buffer, int offset)        {            string json = Newtonsoft.Json.JsonConvert.SerializeObject(data);            return Encoding.UTF8.GetBytes(json, 0, json.Length, buffer, offset);        }        public override int Serialize(object data, FormaterBuffer buffer, int offset)        {            string json = Newtonsoft.Json.JsonConvert.SerializeObject(data);            return Encoding.UTF8.GetBytes(json, 0, json.Length, buffer.Array, offset);        }        public override object Deserialize(Type type, FormaterBuffer buffer, int offset, int count)        {            string value =http://www.mamicode.com/ Encoding.UTF8.GetString(buffer.Array, offset, count);            return Newtonsoft.Json.JsonConvert.DeserializeObject(value, type);        }    }
  • protobuf
     public class ProtobufFormater:FormaterBase    {        public override int Serialize(object data, FormaterBuffer buffer, int offset)        {            buffer.Seek(offset);            ProtoBuf.Meta.RuntimeTypeModel.Default.Serialize(buffer.Stream, data);            return (int)buffer.Stream.Position;        }        public override object Deserialize(Type type, FormaterBuffer buffer, int offset, int count)        {            buffer.Stream.SetLength(count + offset);            buffer.Seek(offset);            return ProtoBuf.Meta.RuntimeTypeModel.Default.Deserialize(buffer.Stream, null, type);        }    }

leveldb.net的一些简单性能改造

虽然leveldb.net只以win dll的基础上包装,但在包装过程的确有些方法针对我个人来说做得并不理想,主要体现在buffer复用方面.其实get,set方法都存在这情况.

 

 /// <summary>        /// Set the database entry for "key" to "value".          /// </summary>        public void Put(byte[] key, byte[] value, WriteOptions options)        {            IntPtr error;            LevelDBInterop.leveldb_put(this.Handle, options.Handle, key, (IntPtr)key.Length, value, (IntPtr)value.LongLength, out error);            LevelDBException.Check(error);            GC.KeepAlive(options);            GC.KeepAlive(this);        }        public unsafe byte[] Get(byte[] key, ReadOptions options)        {            IntPtr error;            IntPtr lengthPtr;            var valuePtr = LevelDBInterop.leveldb_get(this.Handle, options.Handle, key, (IntPtr)key.Length, out lengthPtr, out error);            LevelDBException.Check(error);            if (valuePtr == IntPtr.Zero)                return null;            try            {                var length = (long)lengthPtr;                var value = http://www.mamicode.com/new byte[length];                var valueNative = (byte*)valuePtr.ToPointer();                for (long i = 0; i < length; ++i)                    value[i] = valueNative[i];                return value;            }            finally            {                LevelDBInterop.leveldb_free(valuePtr);                GC.KeepAlive(options);                GC.KeepAlive(this);            }        }

两上个方法都不支持从外部带入buffer的情况,当需要高并发操作的情况而对象序列化内容又比较大的情况下,那的确是会让人感觉到不满意.所以在这基础上添加了一些有利于buffer复用的方法来支持高并发操作下的性能需要.

 

 public void Put(byte[] key, byte[] value, int offset, int length, WriteOptions options)        {            IntPtr error;            LevelDBInterop.leveldb_put(this.Handle, options.Handle, key, (IntPtr)key.Length, value, (IntPtr)length, out error);            LevelDBException.Check(error);            GC.KeepAlive(options);            GC.KeepAlive(this);        }        public unsafe long Get(byte[] key, byte[] buffer, ReadOptions options)        {            IntPtr error;            IntPtr lengthPtr;            var valuePtr = LevelDBInterop.leveldb_get(this.Handle, options.Handle, key, (IntPtr)key.Length, out lengthPtr, out error);            LevelDBException.Check(error);            if (valuePtr == IntPtr.Zero)                return 0;            try            {                var length = (long)lengthPtr;                var valueNative = (byte*)valuePtr.ToPointer();                Marshal.Copy((IntPtr)valuePtr, buffer, 0, (int)length);                return length;            }            finally            {                LevelDBInterop.leveldb_free(valuePtr);                          }        }

leveldb.net对象读写封装


联系
我们