前言:dbcached 1.0 beta* 在 Memcached 1.2.4 的基础上编写而成,也是我的第一个开源C项目。编写 dbcached 的目的是为了最大限度的发挥 Memcached 内存缓存的优势,便捷地维护 Memcached 服务器节点哈希列表,智能地支持 Memcached 故障转移,同时保证数据的持久化存储。

  dbcached

  协议:New BSD License
  作者:张宴
  网址:http://code.google.com/p/dbcached/

  dbcached 是什么?

  ● dbcached 是一款基于 Memcached 和 NMDB 的分布式 key-value 数据库内存缓存系统。

  ● dbcached = Memcached + 持久化存储管理器 + NMDB 客户端接口

  ● Memcached 是一款高性能的,分布式的内存对象缓存系统,用于在动态应用中减少数据库负载,提升访问速度。

  ● NMDB 是一款多协议网络数据库(dbm类)管理器,它由内存缓存和磁盘存储两部分构成,使用 QDBM 或 Berkeley DB 作为后端数据库。

  ● QDBM 是一个管理数据库的例程库,它参照 GDBM 为了下述三点而被开发:更高的处理速度,更小的数据库文件大小,和更简单的API。QDBM 读写速度比 Berkeley DB 要快,详细速度比较见《Report of Benchmark Test》。

  


  Memcached 和 dbcached 在功能上一样吗?

  ● 兼容:Memcached 能做的,dbcached 都能做。除此之外,dbcached 还将“Memcached、持久化存储管理器、NMDB 客户端接口”在一个程序中结合起来,对任何原有 Memcached 客户端来讲,dbcached 仍旧是个 Memcached 内存对象缓存系统,但是,它的数据可以持久存储到本机或其它服务器上的 QDBM 或 Berkeley DB 数据库中。

  ● 性能:前端 dbcached 的并发处理能力跟 Memcached 相同;后端 NMDB 跟 Memcached 一样,采用了libevent 进行网络IO处理,拥有自己的内存缓存机制,性能不相上下。

  ● 写入:当“dbcached 的 Memcached 部分”接收到一个 set(add/replace/...) 请求并储存 key-value 数据到内存中后,“dbcached 持久化存储管理器”能够将 key-value 数据通过“NMDB 客户端接口”保存到 QDBM 或 Berkeley DB 数据库中。

  ● 速度:如果加上“-z”参数,采用 UDP 协议“只发送不接收”模式将 set(add/replace/...) 命令写入的数据传递给 NMDB 服务器端,对 Memcache 客户端写速度的影响几乎可以忽略不计。在千兆网卡、同一交换机下服务器之间的 UDP 传输丢包率微乎其微。在命中的情况下,读取数据的速度跟普通的 Memcached 无差别,速度一样快。

  ● 读取:当“dbcached 的 Memcached 部分”接收到一个 get(incr/decr/...) 请求后,如果“dbcached 的 Memcached 部分”查询自身的内存缓存未命中,则“dbcached 持久化存储管理器”会通过“NMDB 客户端接口”从 QDBM 或 Berkeley DB 数据库中取出数据,返回给用户,然后储存到 Memcached 内存中。如果有用户再次请求这个 key,则会直接从 Memcached 内存中返回 Value 值。

  ● 持久:使用 dbcached,不用担心 Memcached 服务器死机、重启而导致数据丢失。

  ● 变更:使用 dbcached,即使因为故障转移,添加、减少 Memcached 服务器节点而破坏了“key 信息”与对应“Memcached 服务器”的映射关系也不怕。

  ● 分布:dbcached 和 NMDB 既可以安装在同一台服务器上,也可以安装在不同的服务器上,多台 dbcached 服务器可以对应一台 NMDB 服务器。

  ● 特长:dbcached 对于“读”大于“写”的应用尤其适用。


  1. dbcached

  Installation (安装)
引用
wget http://www.monkey.org/~provos/libevent-1.3e.tar.gz
tar zxvf libevent-1.3e.tar.gz
cd libevent-1.3e/
./configure --prefix=/usr
make && make install
cd ../

wget http://dbcached.googlecode.com/files/dbcached-1.0.beta2.tar.gz
tar zxvf dbcached-1.0.beta2.tar.gz
cd dbcached-1.0.beta2/
./configure --prefix=/usr/local/dbcached --with-libevent=/usr
make && make install
cd ../


  Run as a daemon (作为守护进程运行)
引用
/usr/local/dbcached/bin/memcached -d -m 256 -p 11211 -c 51200 -u nobody -x 192.168.0.2 -y 26010 -z 26010


  ● -x nmdb 服务器的域名或者IP地址,推荐使用IP地址

  ● -y <端口号> nmdb 服务器的TCP端口号 (默认: 26010) 支持 set/delete/... 等写命令 和 get 等读命令

  ● -z <端口号> nmdb 服务器的UDP端口号 (默认: 26010) 只支持 get 等都命令, 当使用 -z 参数时,将使用 UDP 协议代替 TCP 协议执行 set 操作,执行 get 操作时仍然使用 TCP 协议。强烈推荐加上 -z 参数。
  ● 其他参数跟 memcached 1.2.4 完全一样,就不再详细说明。

  ● 如果想让 dbcached 通过 NMDB 保存数据时采用 TCP 协议,去掉 -z 参数即可,例如:(除非因防火墙、NAT穿透等问题导致 UDP 协议不可用,否则不建议使用 TCP 协议)
引用
/usr/local/dbcached/bin/memcached -d -m 256 -p 11211 -c 51200 -u nobody -x 192.168.0.2 -y 26010


  ● 如果想让 dbcached 作为普通的 Memcached 运行,去掉 -x、-y、-z 参数即可,例如:
引用
/usr/local/dbcached/bin/memcached -d -m 256 -p 11211 -c 51200 -u nobody



  2. QDBM & NMDB

  QDBM 和 NMDB 均为原版,可以从它们的官方网站下载最新版本。

  QDBM Installation (安装)
引用
wget http://qdbm.sourceforge.net/qdbm-1.8.77.tar.gz
tar zxvf qdbm-1.8.77.tar.gz
cd qdbm-1.8.77/
./configure --prefix=/usr
make
make install
cd ../


  NMDB Installation (安装)
引用
wget http://www.monkey.org/~provos/libevent-1.3e.tar.gz
tar zxvf libevent-1.3e.tar.gz
cd libevent-1.3e/
./configure --prefix=/usr
make && make install
cd ../

wget http://auriga.wearlab.de/~alb/nmdb/files/0.21/nmdb-0.21.tar.gz
tar zxvf nmdb-0.21.tar.gz
cd nmdb-0.21/
make BACKEND=qdbm ENABLE_TIPC=0 ENABLE_SCTP=0 install
cd ../


  Run as a daemon (作为守护进程运行)
引用
/usr/local/bin/nmdb -d /var/dbcached.db -t 26010 -T 192.168.0.2 -u 26010 -U 192.168.0.2 -c 1024

  ● -d 数据库路径(这里使用比 Berkeley DB 更快的 QDBM 数据库),例如 /var/dbcached.db

  ● -t TCP 监听端口 (默认:26010)

  ● -T TCP 监听地址 (默认:任何地址)

  ● -u UDP 监听端口 (默认:26010)

  ● -U UDP 监听地址 (默认:任何地址)

  ● -c 最大的缓存对象数目,单位为千 (默认:128)





技术大类 » Cache与存储 | 评论(39) | 引用(0) | 阅读(54723)
cache
2010-3-24 23:07
区别在于并发性能和速度:1、并发:千万级数据查询,MySQL的速度比较慢,如果并发连接达到300以上,MySQL基本上就崩溃了。就算是只有几万条数据的定长表,撑死也就能达到3000的并发连接。2、速度:往QDBM中写入100万条记录,只需耗费1~4秒,Berkeley DB 只需3~11秒,取平均值5秒计算,每秒可以插入20万条记录,而读取速度还要稍快一点,对于千万级的数据处理性能没问题。而MySQL每秒只能插入50~1000条数据。这个速度还是有点怀疑的,如果真的是文件顺序读写,还是可以的,但常规情况下,应该是随机读写的居多,能达到500/s就不错了。
king
2011-3-23 11:44
这个项目目前还在维护开发吗? 另外问下 假设memecache开的内存为4G 实际后端持久化 能存储多余4G的数据吗 假设缓存数据不设置过期时间 直接依赖先进先出 满了就把早期数据顶出去的方式 memcache中已经因为容量问题被顶出的缓存 是否持久化中还保存?
Deeka Email Homepage
2011-4-2 14:22
dbcached对nmdb版本有严格要求?nmdb2.1编译报错,我换了nmdb2.3,编译通过,但发现dbcached的数据只保存在内存,并没有保存至nmdb。
Deeka Email Homepage
2011-4-2 15:01
nmdb2.1编译通过后,往缓存里写数据,nmdb直接挂掉。囧!
louis vuitton uk Email Homepage
2011-11-23 08:58
This louis vuitton uk for sale belongs to the sounding just what are termed as Louis Vuitton vintage best sellers, many other products and services for the reason that range appearing companies.You will easily notice the unfold zippers of this coach outlet store online. That is the decoration. There are some inside pockets for you as well. They are easy to match your clothes and to carry.Let us inspire your inner beauty with fine christian louboutin sale. Purse the elegance in bridal wedding. Enjoy the fashion.
coach factory outlet Email Homepage
2012-5-17 11:22
Today, following half a century, mentor leather-bases coach factory outlet continues to be the delicate craft of leather-based master is accountable for,Would you like to meet more friends, or go with the times? If yes, coach factory online is opening welcome doors to you.in the market you definitely can find various colorways that are designed in as well as the high quality that applied in. For most of you would like to come. So just come to our coach factory outlet online store to choose one.
louis vuitton sale Email Homepage
2012-5-17 11:22
Louis Vuitton belt at louis vuitton sale is one kind of fashion accessory with high cost performance among the Louis Vuitton accessories.Offering quality LV products with favorable prices, louis vuitton outlet store is at your service. Hurry up, or you can not seize the chance.in fact, louis vuitton is one of the most famous fashion design master.he opened the fist suitcase shop called after his name.
coach outlet online Email Homepage
2012-5-17 11:22
I heard of coach outlet online through the advertisement when I was shopping. And now I often brow the webpage and buy Coach bags online.It is a symbol regarding position not to mention nature.Here I would like to launch a excellent bags pertaining to business men.Which may be coach outlet store.coach outlet has always been simple,durable style features to win consumers.The products are more flexible,with easy bleaching,wear characteristics,and simply use a damp cloth.
coach outlet, Email Homepage
2012-5-17 11:23
Remember the coach outlet provide coach bags which won't be deteriorated into its overall styles by any means. It will maintain its looks, colors, and uniqueness for long time.coach outlet store online has been voted by Hour Detroit magazine readers as the Best of Detroit in their 12th annual readers'poll.Lots of women like which usually amount normally include a coach outlet online ,it provides coziness to many girls that don't even think it is a great bushel of great interest directly to them.
xujie777 Email
2012-5-18 16:25
We aim to make all our customers satisfy with our products. You will find a variety of Men's fashion louis vuitton uk, fashion Women's cheap Louis Vuitton bags in our store at affordable price.Thinking of interesting ways to cost a milestone birthday? louis vuitton online shop had one of the most distinctive distinctive celebrations.bakery along with living room operated by means of about three moment louis vuitton online Most effective Pastry Chef’s of the year Rammy Nominee Chef’s.
xujie777 Email
2012-5-18 16:25
The choices are likely to be basically countless seeing that louis vuitton outlet occurs with the help of completely new and also incredible concepts once in a while.Louis vuitton Wholesale Monogram Canvas HandbagsLouis Vuitton Collection Beach Handbags louis vuitton bags outlet Damier Canvas HandbagsLouis vuitton Mahina HandbagsLouis Vuitton Monogram Mini Lin HandbagsLouis Vuitton Monogram Multicolore HandbagsLouis vuitton Monogram Vernis HandbagsLouis Vuitton Wholesale Epi Leather HandbagsLouis Vuitton For Men HandbagsLouis Vuitton Damier Canvas WalletsLouis Vuitton Epi Leather WalletsLouis Vuitton Monogram Canvas WalletsLouis Vuitton Monogram Vernis WalleLouis Vuitton ShoesLouis Vuitton Men wallets.As the Authentic Louis Vuitton are so high-priced, so came the louis vuitton handbags outlet.
xujie777 Email
2012-5-18 16:25
coach outlet has become a popular shopping experience for consumers around the world, and a desirable distribution channel for manufacturer's and retailers.Getting your hands on coach outlet store online can be a hefty investment of hundreds of dollars.But do not despair,the Coach Outlet store could be the answer to your prayers.coach outlet online is your smart choice when you want to get the discount Coach accessory. You can find the exact Coach Bags and other accessory you want at a low price that's right for you. My dear friends, let the coach outlet online pave your way into the world of high fashion with their ultimate fashion factory.
xujie555
2012-5-21 13:41
coach factory outlet uses graceful accessories to match the classical logo of coach, which is the best combination of coach. The handmade coach products?can make you more charming and graceful.coach factory online provides people many coach goods. If you wish to snatch the coach handbag, then this best method is made for that you like for coach discount.coach factory outlet online is a fashion brand to ensure its quality. With designer coach shoulder bags, you will always attract people's attention. The bags will emphasize your personal style and taste.<br/>
xujie333
2012-5-21 15:01
coach outlet online Store guarantee that all the coach handbags offered are own high quality. In addition , all of them are sold at an unexpected low price.If you want to purchase, just visit their website.coach factory outlet is really sizzling kinds of shopping way for you. With the usage of the replica designer coach bags, you can surely be able to change your individual looks in a stunning manner.Coach bags on sale from the coach outlet are cheap or discount prices that you certainly will stand out from the crowd on your next camping trip!<br/>
coach88888888 Email
2012-8-13 17:46
If you desire to go to Coach Factory Outlet, but have no idea in which to go, you can research online. It is no doubt that there is drastically information and details about it for the reference.Coach Factory Online are loved by many people, when you walk in the street, you could see many people take coach styles.hat experts claim Coach Outlet Online shopping is in the changes they are available in.Coach Outlet will send you a coupon in the post to use in their upon only after you type a achieve. The Coach bags are customarily somewhat many than the ones in the stores.
django Email Homepage
2013-11-30 18:57
Uggs Sale the ideal idea Coach Outlet Online  enough time to search for many of reduction Ugg boot. Like big portions? If you began ready your apartment, Coach Outlet  it's always best to consider the most effective way direct to the stage it could be to utilise your inventory throughout Coach Factory Outlet  concern, or maybe you basically won't destination stuff not enough.we're anti- whitening tray because Coach Outlet Online  i will be enjoy just get germ dog breeders
分页: 2/2 第一页 上页 1 2 最后页
发表评论
表情
emotemotemotemotemot
emotemotemotemotemot
emotemotemotemotemot
emotemotemotemotemot
emotemotemotemotemot
打开HTML
打开UBB
打开表情
隐藏
记住我
昵称   密码   游客无需密码
网址   电邮   [注册]