How to decide if a product is right for your forum

[Floren]

Is been a while I deal with large forums and I'm always amazed how many products I see installed into each one of them. To avoid repeating myself again, I decided to write this little tutorial that can help you determine if a product is right for your forum. The last thing you want is to have down time because you installed a product who crippled your servers for hours...

Let's presume you are an unexperienced person, related to PHP and MySQL but an excellent manager with a successful forum. You go to vBulletin.org and start browsing for new products that might be interesting... until you spot some "cool" feature. You read about it and get excited, that is exactly what you wanted to implement for a while in your forums and it was not available! Hooray! You install it without hesitation and announce it to your users, so they can use the new feature also.

One day later, your server admin is calling you asking you what is going on and if you changed anything to the server configuration or installed some new software. Curious, you ask why and you let him know that you added a new product that you desired to have it implemented for a while... To your total shock, you find out that the product is quadrupling the overall server load.

"OK, is not the end of the world, I will uninstall it and apologize to users..."
You go to your admin CP and try to uninstall the product. Something is wrong, some printed text flash fast in front of your eyes and you don't know what is going on... In the same time, you go visit the forums and you notice everything runs slower then before, the server load is rising again... "In the name of God, what I did wrong?"

You did not do anything wrong. In fact, you are one of the many people who trust vBulletin.org as being the perfect coded universe of vBulletin applications. Unfortunately, many of products found there are badly written, creating more chaos into your forums then actually helping.

I have this rule of the thumb: Never install a product, unless it is absolutely necessary.
vBulletin was designed to cover all your forum needs. The software was tested through a large combination of servers, browsers and busy sites with over 3,000 online users. So if a bug is present, it will make surface immediately, into RC releases.

On the other hand... you see very rarely a product that is in Beta, even if the administration is offering you this option. As developer, if you do not have the resources to test your software into an emulated environment with over 1,000 online users and run it through several operating systems with various browsers installed, you cannot directly release it to the public.

So how do you decide if the product is written well and is right for you?
After all, you are not a code freak... you have a business to manage: your forum.

It is very easy actually, follow these 3 simple steps:

1. Ask the product developer if the product was benchmarked. Request more information about the server load monitoring into a live site and let the developer to provide stats and benchmarks of the site with the product installed. A good example would be:

"Hi Floren, could you be kind and let me know some server stats with your product enabled? I would like to know if you can run quickly a http_load benchmark on your localhost. It should not take you more then 15minutes but it is important know this information before I install your product. Let me know what are the before/after results... Thanks."

If the developer took the time to develop a product properly and test it well, he will be able to provide you with stats like:

Product Enabled
500 online users, 10 seconds parallel test

269 fetches, 500 max parallel, 5.34661e+06 bytes, in 10.0094 seconds
19875.9 mean bytes/connection
26.8747 fetches/sec, 534159 bytes/sec
msecs/connect: 0.904112 mean, 85.94 max, 0.074 min
msecs/first-response: 3783.43 mean, 9272.46 max, 1.373 min

1000 online users, 10 seconds parallel test

263 fetches, 1000 max parallel, 5.32541e+06 bytes, in 10.0039 seconds
20248.7 mean bytes/connection
26.2897 fetches/sec, 532332 bytes/sec
msecs/connect: 0.824681 mean, 78.576 max, 0.071 min
msecs/first-response: 3868.94 mean, 8879.29 max, 165.277 min

Product Disabled
500 online users, 10 seconds parallel test

779 fetches, 500 max parallel, 1.564e+07 bytes, in 10.0024 seconds
20077 mean bytes/connection
77.8815 fetches/sec, 1.56363e+06 bytes/sec
msecs/connect: 1.50359 mean, 131.5 max, 0.07 min
msecs/first-response: 1674.58 mean, 8914.71 max, 97.031 min

1000 online users, 10 seconds parallel test

633 fetches, 1000 max parallel, 1.26093e+07 bytes, in 10.0019 seconds
19919.9 mean bytes/connection
63.2879 fetches/sec, 1.26069e+06 bytes/sec
msecs/connect: 124.028 mean, 3007.64 max, 0.073 min
msecs/first-response: 1971.85 mean, 8686.33 max, 1.099 min

Test Comparison

OPTIONS		Product Disabled	Product Enabled
-----------------------------------------------------------------
Users		1000	500		1000	500
-----------------------------------------------------------------
Fetches		633	779		263	269
Fetches/sec	63.28	77.88		26.28	26.87

Now, you can clearly see that on a busy site environment, the server performance is degraded. That results into a higher server load, 3 times more intensive compared when the product was disabled. Any developer should be able to provide you with this type of benchmarks, no matter how primitive they are. That proves he/she did his/her job and developed a responsible product. He/she is letting you know this product is not designed for busy sites.

2. Ask other forum members if the product was tested into a forum with over x,xxx users online. A good feedback from few fellow admins is aways welcome. It will save you the trouble to restore 10GB of data, in case the product breaks everything due to lack of testing or limited coding abilities. If you see that other users confirm they did not have any issues with 1,500 users or more, then we have another green flag in our hands to install the product.

3. Ask a PHP consultant to evaluate the product throughly, if it's something that requires database modifications. He/she will know how to determine if future vBulletin upgrades might break the current product functionality. You might say that everything is OK and the benchmarks are great while the server resources are not affected... What if your product is not working anymore, when you upgrade your forum? There is nothing more frustrating then to have your users getting used to a feature that is not available anymore, because the product was abandoned.

A good sign that will let you know a product was not designed properly is the necessity to upgrade the product with almost every new vBulletin release. Most particular are the products who require modifications to important tables (post, thread, forum, etc.). Those type of products will have 90% chances they will not work properly when a new vBulletin version is released.

Have you noticed that all vBulletin products interact with each others very well? You do not have to upgrade the vBulletin Blog, if you upgrade the forums... Yes, it costs money to hire a consultant. But would you not be more happy to know that your installed product will not create any issues in the long run? Personally, I think is worth the investment.

But no matter what, make sure you install the product into a test forum first, where you have all current products installed with a fresh database. You will be able to test everything and verify if all options work as announced, while merged with your currently installed products. Invite your moderators to test also the product features, so they can report any errors.

Follow these 3 rules and you will protect well your investment, while allowing users to enjoy new features.