首页 > 代码库 > memcached基础

memcached基础

1.1、memcached是什么?

    memcached 是以LiveJournal 旗下Danga Interactive 公司的Brad Fitzpatric 为首开发的一款软件。现在已成为 mixi、 hatena、 Facebook、 Vox、LiveJournal等众多服务中 提高Web应用扩展性的重要因素。

    许多Web应用都将数据保存到RDBMS中,应用服务器从中读取数据并在浏览器中显示。 但随着数据量的增大、访问的集中,就会出现RDBMS的负担加重、数据库响应恶化、 网站显示延迟等重大影响。

    这时就该memcached大显身手了。memcached是高性能的分布式内存缓存服务器。 一般的使用目的是,通过缓存数据库查询结果,减少数据库访问次数,以提高动态Web应用的速度、 提高可扩展性。

1.2、memcached特性

  • 1.2.1 协议简单
  • 1.2.2 基于libevent的事件处理
  • 1.2.3 内置内存存储方式
  • 1.2.4 memcached不互相通信的分布式

1.2.1、协议简单

    memcached的服务器客户端通信并不使用复杂的XML等格式,而使用简单的基于文本行的协议。因此,通过telnet 也能在memcached上保存数据、取得数据。下面是例子。

$ telnet localhost 11211Trying 127.0.0.1...Connected to localhost.Escape character is ^].get fooVALUE foo 0 2hiENDstatsSTAT pid 8861(etc)

    协议文档位于memcached的源代码内,也可以参考以下的URL。

     http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt

1.2.2、基于libevent的事件处理

    libevent是个程序库,它将Linux的epoll、BSD类操作系统的kqueue等事件处理功能封装成统一的接口。即使对服务器的连接数增加,也能发挥O(1)的性能。 memcached使用这个libevent库,因此能在Linux、BSD、Solaris等操作系统上发挥其高性能。

    关于事件处理这里就不再详细介绍,可以参考Dan Kegel的The C10K Problem。

1.2.3、内置内存存储方式

    为了提高性能,memcached中保存的数据都存储在memcached内置的内存存储空间中。 由于数据仅存在于内存中,因此重启memcached、重启操作系统会导致全部数据消失。 另外,内容容量达到指定值之后,就基于LRU(Least Recently Used)算法自动删除不使用的缓存。 memcached本身是为缓存而设计的服务器,因此并没有过多考虑数据的永久性问题。

1.2.4、memcached不互相通信的分布式

    memcached尽管是“分布式”缓存服务器,但服务器端并没有分布式功能。

    各个memcached不会互相通信以共享信息。那么,怎样进行分布式呢? 这完全取决于客户端的实现。

1.3、安装memcached

1.3.1、安装memcached

     运行memcached前需要本文开头介绍的libevent库。安装libevent如下:

[root@localhost ~]# tar -zxvf libevent-2.0.21-stable.tar.gz...[root@localhost ~]# cd libevent-2.0.21-stable/[root@localhost libevent-2.0.21-stable]# ./configure...[root@localhost libevent-2.0.21-stable]# make...[root@localhost libevent-2.0.21-stable]# make install...

    安装 memcached,下载memcached,安装如下:

[root@localhost ~]# tar -xzvf memcached-1.4.20.tar.gz ...[root@localhost ~]# cd memcached-1.4.20/[root@localhost memcached-1.4.20]# ./configure...[root@localhost memcached-1.4.20]# make...[root@localhost memcached-1.4.20]# make install...

测试安装是否成功

[root@localhost ~]# /usr/local/bin/memcached -h/usr/local/bin/memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory[root@localhost ~]# ldd /usr/local/bin/memcached         linux-vdso.so.1 =>  (0x00007fffd5b69000)        libevent-2.0.so.5 => not found        librt.so.1 => /lib64/librt.so.1 (0x0000003a44600000)        libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003a43e00000)        libc.so.6 => /lib64/libc.so.6 (0x0000003a43a00000)        /lib64/ld-linux-x86-64.so.2 (0x0000003a43200000)[root@localhost ~]# LD_DEBUG=libs /usr/local/bin/memcached -v      7715:     find library=libevent-2.0.so.5 [0]; searching      7715:      search cache=/etc/ld.so.cache      7715:      search path=/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64                (system search path)      7715:       trying file=/lib64/tls/x86_64/libevent-2.0.so.5      7715:       trying file=/lib64/tls/libevent-2.0.so.5      7715:       trying file=/lib64/x86_64/libevent-2.0.so.5      7715:       trying file=/lib64/libevent-2.0.so.5      7715:       trying file=/usr/lib64/tls/x86_64/libevent-2.0.so.5      7715:       trying file=/usr/lib64/tls/libevent-2.0.so.5      7715:       trying file=/usr/lib64/x86_64/libevent-2.0.so.5      7715:       trying file=/usr/lib64/libevent-2.0.so.5      7715:     /usr/local/bin/memcached: error while loading shared libraries: libevent-2.0.so.5: cannot open shared object file: No such file or directory[root@localhost ~]# whereis libevent-2.0.so.5libevent-2.0.so: /usr/local/lib/libevent-2.0.so.5[root@localhost ~]# /usr/local/bin/memcached -h[root@localhost ~]# /usr/local/bin/memcached -hmemcached 1.4.20-p <num>      TCP port number to listen on (default: 11211)-U <num>      UDP port number to listen on (default: 11211, 0 is off)-s <file>     UNIX socket path to listen on (disables network support)-A            enable ascii "shutdown" command-a <mask>     access mask for UNIX socket, in octal (default: 0700)-l <addr>     interface to listen on (default: INADDR_ANY, all addresses)              <addr> may be specified as host:port. If you dont specify              a port number, the value you specified with -p or -U is              used. You may specify multiple addresses separated by comma              or by using -l multiple times-d            run as a daemon-r            maximize core file limit-u <username> assume identity of <username> (only when run as root)-m <num>      max memory to use for items in megabytes (default: 64 MB)-M            return error on memory exhausted (rather than removing items)-c <num>      max simultaneous connections (default: 1024)-k            lock down all paged memory.  Note that there is a              limit on how much memory you may lock.  Trying to              allocate more than that would fail, so be sure you              set the limit correctly for the user you started              the daemon with (not for -u <username> user;              under sh this is done with ulimit -S -l NUM_KB).-v            verbose (print errors/warnings while in event loop)-vv           very verbose (also print client commands/reponses)-vvv          extremely verbose (also print internal state transitions)-h            print this help and exit-i            print memcached and libevent license-P <file>     save PID in <file>, only used with -d option-f <factor>   chunk size growth factor (default: 1.25)-n <bytes>    minimum space allocated for key+value+flags (default: 48)-L            Try to use large memory pages (if available). Increasing              the memory page size could reduce the number of TLB misses              and improve the performance. In order to get large pages              from the OS, memcached will allocate the total item-cache              in one large chunk.-D <char>     Use <char> as the delimiter between key prefixes and IDs.              This is used for per-prefix stats reporting. The default is              ":" (colon). If this option is specified, stats collection              is turned on automatically; if not, then it may be turned on              by sending the "stats detail on" command to the server.-t <num>      number of threads to use (default: 4)-R            Maximum number of requests per event, limits the number of              requests process for a given connection to prevent               starvation (default: 20)-C            Disable use of CAS-b            Set the backlog queue limit (default: 1024)-B            Binding protocol - one of ascii, binary, or auto (default)-I            Override the size of each slab page. Adjusts max item size              (default: 1mb, min: 1k, max: 128m)-F            Disable flush_all command-o            Comma separated list of extended or experimental options              - (EXPERIMENTAL) maxconns_fast: immediately close new                connections if over maxconns limit              - hashpower: An integer multiplier for how large the hash                table should be. Can be grown at runtime if not big enough.                Set this based on "STAT hash_power_level" before a                 restart.              - tail_repair_time: Time in seconds that indicates how long to wait before                forcefully taking over the LRU tail item whose refcount has leaked.                The default is 3 hours.              - hash_algorithm: The hash table algorithm                default is jenkins hash. options: jenkins, murmur3              - lru_crawler: Enable LRU Crawler background thread              - lru_crawler_sleep: Microseconds to sleep between items                default is 100.              - lru_crawler_tocrawl: Max items to crawl per slab per run                default is 0 (unlimited)

1.3.2、启动memcached

    从终端输入以下命令,启动memcached

[root@localhost ~]# /usr/local/bin/memcached -p 11211 -m 64m -u root -vvslab class   1: chunk size        96 perslab   10922slab class   2: chunk size       120 perslab    8738slab class   3: chunk size       152 perslab    6898slab class   4: chunk size       192 perslab    5461slab class   5: chunk size       240 perslab    4369slab class   6: chunk size       304 perslab    3449slab class   7: chunk size       384 perslab    2730slab class   8: chunk size       480 perslab    2184slab class   9: chunk size       600 perslab    1747slab class  10: chunk size       752 perslab    1394slab class  11: chunk size       944 perslab    1110slab class  12: chunk size      1184 perslab     885slab class  13: chunk size      1480 perslab     708slab class  14: chunk size      1856 perslab     564slab class  15: chunk size      2320 perslab     451slab class  16: chunk size      2904 perslab     361slab class  17: chunk size      3632 perslab     288slab class  18: chunk size      4544 perslab     230slab class  19: chunk size      5680 perslab     184slab class  20: chunk size      7104 perslab     147slab class  21: chunk size      8880 perslab     118slab class  22: chunk size     11104 perslab      94slab class  23: chunk size     13880 perslab      75slab class  24: chunk size     17352 perslab      60slab class  25: chunk size     21696 perslab      48slab class  26: chunk size     27120 perslab      38slab class  27: chunk size     33904 perslab      30slab class  28: chunk size     42384 perslab      24slab class  29: chunk size     52984 perslab      19slab class  30: chunk size     66232 perslab      15slab class  31: chunk size     82792 perslab      12slab class  32: chunk size    103496 perslab      10slab class  33: chunk size    129376 perslab       8slab class  34: chunk size    161720 perslab       6slab class  35: chunk size    202152 perslab       5slab class  36: chunk size    252696 perslab       4slab class  37: chunk size    315872 perslab       3slab class  38: chunk size    394840 perslab       2slab class  39: chunk size    493552 perslab       2slab class  40: chunk size    616944 perslab       1slab class  41: chunk size    771184 perslab       1slab class  42: chunk size   1048576 perslab       1<26 server listening (auto-negotiate)<27 server listening (auto-negotiate)<28 send buffer was 262144, now 268435456<28 server listening (udp)<29 server listening (udp)<32 send buffer was 262144, now 268435456<32 server listening (udp)<30 server listening (udp)<33 server listening (udp)<34 server listening (udp)<31 server listening (udp)<35 server listening (udp)

    这里显示了调试信息。这样就在前台启动了memcached,监听TCP端口11211 最大内存使用量为64M。调试信息的内容大部分是关于存储的信息。

    作为daemon后台启动时,只需 

     #/usr/local/bin/memcached -p 11211 -m 64m -d -u root

1.4、客户端连接

    许多语言都实现了连接memcached的客户端,其中以Perl、PHP为主。 仅仅memcached网站上列出的语言就有Perl、PHP、Python、Ruby、C#、C/C++、Lua等等。下面介绍通过Java语言操作memcached。

1.5、使用java-memcached

1.5.1、下载Memcached-Java-Client

    下载 Memcached-Java-Client 最新版本,解压缩,获取里面的4个jar文件,加入工程classpath。

1.5.2、编码测试

 1         String[] addr = { "10.108.1.97:11211" }; 2         Integer[] weights = { 3 }; 3         SockIOPool pool = SockIOPool.getInstance(); 4         pool.setServers(addr); 5         pool.setWeights(weights); 6         pool.setInitConn(5); 7         pool.setMinConn(5); 8         pool.setMaxConn(200); 9         pool.setMaxIdle(1000 * 30 * 30);10         pool.setMaintSleep(30);11         pool.setNagle(false);12         pool.setSocketTO(30);13         pool.setSocketConnectTO(0);14         pool.initialize();15 16         MemCachedClient client = new MemCachedClient();17 18         client.set("user", "Jobs");19 20         // 将数据放入缓存,并设置失效时间21         // Date date = new Date(2000000);22         // client.set("test1", "test1", date);23 24         // 删除缓存数据25         // client.delete("test1");26         // client.delete("test2");27         // client.flushAll();28 29         // 获取缓存数据30         System.out.println((String) client.get("user"));

1.5.3、保存数据

    向memcached保存数据的方法有:

    add        仅当存储空间中不存在键相同的数据时才保存

    replace   仅当存储空间中存在键相同的数据时才保存

    set        与add和replace不同,无论何时都保存

1.5.4、获取数据

    获取数据可以使用get和getMulti方法。一次取得多条数据时使用get_multi。get_multi可以非同步地同时取得多个键值,其速度要比循环调用get快数十倍。

1.5.5、删除数据

    删除数据使用delete方法,不过它有个独特的功能。删除第一个参数指定的键的数据。第二个参数指定一个时间值,可以禁止使用同样的键保存新数据。此功能可以用于防止缓存数据的不完整。但是要注意,set 函数忽视该阻塞,照常保存数据。

 

 

参考:http://blog.charlee.li/memcached-001/