Brave New Geek – Introspections of a software engineer. A common problem in messaging middleware is that of efficiently matching message topics with interested subscribers. For example, assume we have a set of subscribers, numbered 1 to 3: Subscriber. Match Request. 1forex. And we have a stream of messages, numbered 1 to N: Message. Topic. 1forex. gbp. Nstock. nasdaq. msft. We are then tasked with routing messages whose topics match the respective subscriber requests, where a “*” wildcard matches any word. This is frequently a bottleneck for message- oriented middleware like Zero. MQ, Rabbit. MQ, Active. MQ, TIBCO EMS, et al. Because of this, there are a number of well- knownsolutionsto the problem. In this post, I’ll describe some of these solutions, as well as a novel one, and attempt to quantify them through benchmarking. ![]() As usual, the code is available on Git. Hub. The Naive Solution. The naive solution is pretty simple: use a hashmap that maps topics to subscribers. Subscribing involves adding a new entry to the map (or appending to a list if it already exists). Matching a message to subscribers involves scanning through every entry in the map, checking if the match request matches the message topic, and returning the subscribers for those that do. Inserts are approximately O(1) and lookups approximately O(n*m) where n is the number of subscriptions and m is the number of words in a topic. This means the performance of this solution is heavily dependent upon how many subscriptions exist in the map and also the access patterns (rate of reads vs. Since most use cases are heavily biased towards searches rather than updates, the naive solution—unsurprisingly—is not a great option. The microbenchmark below compares the performance of subscribe, unsubscribe, and lookup (matching) operations, first using an empty hashmap (what we call cold) and then with one containing 1,0. With the populated subscription map, lookups are about three orders of magnitude slower, which is why we have to use a log scale in the chart below. Inverted Bitmap. The inverted bitmap technique builds on the observation that lookups are more frequent than updates and assumes that the search space is finite. Consequently, it shifts some of the cost from the read path to the write path. It works by storing a set of bitmaps, one per topic, or criteria, in the search space. Subscriptions are then assigned an increasing number starting at 0. We analyze each subscription to determine the matching criteria and set the corresponding bits in the criteria bitmaps to 1. For example, assume our search space consists of the following set of topics: forex. ![]() Hello, I'm Geordie and I currently work at ADX Portland running the Laser Cutter and Engraver. My job is to take in customer's projects, set them up and run the. Column-Stores vs RDBMS in areas of compression, partitioning, etc. and usage of Column-Store benefits in Oracle products. Compatibility. One of the major advantages of TIFF files is that they can be edited with most popular image editing software on the market. All popular operating. We then have the following subscriptions: 0 = forex.* (matches forex. When we index the subscriptions above, we get the following set of bitmaps: Criteria. When we match a message, we simply need to lookup the corresponding bitmap and check the set bits. As we see below, subscribe and unsubscribe are quite expensive with respect to the naive solution, but lookups now fall well below half a microsecond, which is pretty good (the fact that the chart below doesn’t use a log scale like the one above should be an indictment of the naive hashmap- based solution). The inverted bitmap is a better option than the hashmap when we have a read- heavy workload. One limitation is it requires us to know the search space ahead of time or otherwise requires reindexing which, frankly, is prohibitively expensive. Optimized Inverted Bitmap. The inverted bitmap technique works well enough, but only if the topic space is fairly static. It also falls over pretty quickly when the topic space and number of subscriptions are large, say, millions of topics and thousands of subscribers. The main benefit of topic- based routing is it allows for faster matching algorithms in contrast to content- based routing, which can be exponentially slower. The truth is, to be useful, your topics probably consist of stock. We could end up with an explosion of topics and, even with efficient bitmaps, the memory consumption tends to be too high despite the fact that most of the bitmaps are quite sparse. Fortunately, we can reduce the amount of memory we consume using a fairly straightforward optimization. Rather than requiring the entire search space a priori, we simply require the max topic size, in terms of words, e. We can handle topics of the max size or less, e. If we see a message with more words than the max, we can safely assume there are no matching subscriptions. In computing, indexed color is a technique to manage digital images' colors in a limited fashion, in order to save computer memory and file storage, while speeding up.![]() The optimized inverted bitmap works by splitting topics into their constituent parts. Each constituent position has a set of bitmaps, and we use a technique similar to the one described above on each part. We end up with a bitmap for each constituent which we perform a logical AND on to give a resulting bitmap. Each 1 in the resulting bitmap corresponds to a subscription. This means if the max topic size is n, we only AND at most n bitmaps. Furthermore, if we come across any empty bitmaps, we can stop early since we know there are no matching subscribers. Let’s say our max topic size is 2 and we have the following subscriptions: 0 = forex.*1 = stock. The inverted bitmap for the first constituent looks like the following: forex.*stock. And the second constituent bitmap: forex.*stock. The “null” and “other” rows are worth pointing out. Null” simply means the topic has no corresponding constituent. For example, “index” has no second constituent, so “null” is marked. Other” allows us to limit the number of rows needed such that we only need the ones that appear in subscriptions. For example, if messages are published on forex. I merely subscribe to forex.*, there’s no need to index eur, usd, or gbp. Instead, we just mark the “other” row which will match all of them. Let’s look at an example using the above bitmaps. Imagine we want to route a message published on forex. We split the topic into its constituents: “forex” and “eur.” We get the row corresponding to “forex” from the first constituent bitmap, the one corresponding to “eur” from the second (other), and then AND the rows. AND1. 00. 0The forex.* subscription matches. Let’s try one more example: a message published on stock. AND0. 10. 1In this case, we also need to OR the “other” row for the second constituent. This gives us a match for stock. Subscribe operations are significantly faster with the space- optimized inverted bitmap compared to the normal inverted bitmap, but lookups are much slower. However, the optimized version consumes roughly 4. The increased flexibility and improved scalability makes the optimized version a better choice for all but the very latency- sensitive use cases. Trie. The optimized inverted bitmap improves space complexity, but it does so at the cost of lookup efficiency. Is there a way we can reconcile both time and space complexity? While inverted bitmaps allow for efficient lookups, they are quite wasteful for sparse sets, even when using highly compressed bitmaps like Roaring bitmaps. Tries can often be more space efficient in these circumstances. When we add a subscription, we descend the trie, adding nodes along the way as necessary, until we run out of words in the topic. Finally, we add some metadata containing the subscription information to the last node in the chain. To match a message topic, we perform a similar traversal. If a node doesn’t exist in the chain, we know there are no subscribers. One downside of this method is, in order to support wildcards, we must backtrack on a literal match and check the “*” branch as well. For the given set of subscriptions, the trie would look something like the following: forex.*stock. You might be tempted to ask: “why do we even need the “*” nodes? When someone subscribes to stock.*, just follow all branches after “stock” and add the subscriber.” This would indeed move the backtracking cost from the read path to the write path, but—like the first inverted bitmap we looked at—it only works if the search space is known ahead of time. It would also largely negate the memory- usage benefits we’re looking for since it would require pre- indexing all topics while requiring a finite search space. It turns out, this trie technique is how systems like Zero. MQ and Rabbit. MQ implement their topic matching due to its balance between space and time complexity and overall performance predictability. We can see that, compared to the optimized inverted bitmap, the trie performs much more predictably with relation to the number of subscriptions held. Water- Slide Model Decal How- To For Modelers. The main use of this paper is for creating transfer decals for various model trains, planes, cars, etc. Create your own model decals for railroad names, stripes, logos, stained glass windows, road racing emblems, aircraft and military markings, signs, and limitless other custom artworks. Copy any pre- printed design or create your own artwork with ink and paper or a computer drawing program. Duplicate your drawing onto the blank, water- slide decal paper using an office copier, laser printer, alps printer, or ink jet printer. For color copies, take the original artwork/drawing and water- slide, model decal paper to your local copy center. This works like any water decal. Simply cut out the decal (printed/copied image), dip in water, and slide it into place. You can use clear decal paper for most lettering. Use white decal paper when white is your background color. Please note, copiers and printers do NOT print white except for the ALPS 5. Each sheet of model decal paper is 8- 1/2" x 1. If you are using ALPS and printing white with your printer then we suggest our blue decal paper (with clear coat) so that the white print will be easier to see on Blue than on white paper. The Alps MD- 5. 00. However, if you do not have an ALPS printer, you can use our "white" paper that may be used for all artworks that have a white color background. Creating your own Custom Decals has now become easier than ever to do. All you need is a computer and a laser printer. The only task remaining is the preparation of the artwork. Creating decals is fun and easy and you can do it using the word processing software already installed on your computer. Once you have the artwork prepared on the computer, you just print it out onto the decal paper. If you really want perfection or precision, then perhaps you would benefit from for example, an illustration software program that will enable you to match (exactly) prototype lettering. White Lettering/White artwork/White Paper. In order to print the color "white" successfully you will either require an Alps printer which has the ability to print opaque white, or our high performance WHITE DECAL PAPER. If you do not have an Alps printer, then you may order our white decal paper which is especially made for artwork that has white in it. If you do not own an Alps laser printer, you are not completely out of luck. Now you can use our BEL INC white decal paper and get the same excellent results for all of your artworks that include the color white. You may place your order on our online shopping cart now! You can choose to use any of the following software for your custom decal creations. Adobe Illustrator, Macromedia Freehand, Deneba Canvas, or Corel. DRAW. There are some simpler drawing programs that pack fewer widgets at a lower cost. Some of these come pre- installed on Apple i. Macs and many consumer- oriented PCs. Raster vs. Vector. Graphic programs are either raster or vector. Raster art is made from a matrix of colored dots, like what you get from a desktop scanner, painting program, or digital camera. Vector art is a collection of data describing the art, like the coordinates of a line's endpoints or the size, font, and location of any text. Raster files are larger than vector files and take longer to print and often lose some clarity when enlarged. It is suggested to use raster art in bitmap or line art mode, where only black and white is used. If your artwork has multiple colors, you will need to create a separate piece of art for each color. If you want to create water slide decal transfers for more complex, full color art, you can scan it and print it onto blank water- slide decal paper. This decal paper is ready to use with your laser printer, color copier, alps printer, or ink jet printer. It already has the soft decal film coated paper. This water- slide transfer paper is already pre coated with our state- of- the- art clear decal material that is soft yet extremely durable. Our laser paper is specially formulated for use with laser printers like the ALPS printers and many others. Our paper can also be used with several late model personal copiers and laser printers. Our water- slide decal paper has been used by hundreds of hobbyists and model creators over the past decade. BEL INC provides this paper to most of Americas' largest water- slide decal printing companies. Our high quality model decal paper product has been tested and approved by some of the most well known (and critical) model decal illustrators. The overall performance and end results of our paper are superior to those of any other decal paper supplier. BEL INC water- slide model decal paper will open doors and allow you to maximize on your creativity. Print onto our model paper using your laser printer (dye sublimation printer) or ink jet printer the way you would any printer paper. Cut out the image and soak the decal in water for 5- 1. Slide the film off the backing (discard the backing paper) and position the film where you want it on your surface (model plane, train, car, toy, wood, plastic, cardboard, wax candle, ceramic, glass, porcelain, CD's, computer mouse, etc.). Add water to the surface if you need it. Pat the Decal dry. Decals may be coated with a gloss coat or a flat finish to seal the surface. GOOD LUCK and ENJOY !!! How to make your own decals using an Inkjet printer? Getting frustrated about the virtual non- availablility of VNAF national insignia and other markings I decided to try and make my own. Usually the VNAF decals that are available are not in the right colors, or their dimensions and lay- out are wrong. To make your own decals all you need is the following. Imaging software that supports the creation of vector graphics (I use Paint Shop Pro 7), and a good printer (I get great results in any field with my HP Deskjet 9. Cxi). A computer would be great, too. The raw material. Clear and white Inkjet decals sheets, and a spray can of gloss clear paint. I ordered a starters set at http: //www. It contains a spray can of high- gloss Krylon acrylic paint, and two sets of decals sheet containing four clear and four white sheets measuring 2. The package arrived a dazzling three days after I ordered them. Bare in mind that you cannot print on regular decal paper with an Inkjet printer since the Inkjet- ink is water- soluable and WILL bleed on the surface. Only specially coated Inkjet decal paper is suitable. There are several ways to approach this, each having their own advantages and disadvantages. An inkjet printer cannot print white. Therefor any white areas in your decal will have to be printed on wite decal paper. Let's take a VNAF national insignia as an example. First, create the insignia as a vector graphic in your imaging- software. Creating it as a vector graphic instead of a pixel- oriented graphic has the advantage that you can re- scale the graphic to any size, from tiny to giant, without quality- loss, i. This means that you only need to create the image once and can then print decals of it for models in any given scale. Also make sure "Anti- aliasing" is enabled when you create your graphic. Doing so will result in much smoother edges than with it disabled. Create the design bigger than that they will be printed. You can reduce the size when printing by selecting a percentage in the print- preview screen. This will also add to getting smoother edges. Save your work from time to time to prevent a computer crash wasting hours of work. If you have no clue what I am talking about, do a search on the Net using the words "create vector graphics" as a query. There are plenty of good tutorials out there. What I did was create a template of the insignia, so just an outline of the shape, and print that on white decal paper with a very thin, very light- gray outline. Next I created the VNAF insignia in the same size and the right colors. I left the white star transparent since I can't print white. You may put a white start in your decal if you want since your printer will just ignore it. You can buy after- market decal sets with plain white stars and circles and dimension your insignia to fit them, this saves you having to cut out the white background decal from a sheet yourself, but it is questionable you will always find the right size star or circle. The template- decal. The actual decal which is slid on top of the template decal on your model.
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |