TV2 portal is served by well-known Drupal CMS with LAMP environment. Video operations are handled with FFmpeg software. During the work, our designers created more than 200 sketches and design ideas, more than 30 content layouts to provide the best informational architecture for this portal.

Last month we have implemented the parental control feature for our in-house ISP Zing. We have decided to store black and whitelisted data in MySQL database such as URLs, domain names, badwords and so on. This solution is OK for us since it gives simple way to manipulate data (update and change) and simple way to request data to do different checks.

Next, we have implemented SQUID-based ICAP module which implements next functions:

1. checks headers on blacklisted/whitelisted domains, URLs, keywords;
2. checks content for badwords
3. replaces content if required.

We had implemented the module in C/C++ as C-ICAP plugin which works with SQUID 3. All had functioned perfectly but one thing was annoying. We decided to use regular expressions to check content for badwords and we have implemented PCRE library find/replace functions to complete the tasks (by the way, PCRE doesn’t handle replace by default, so we have ported replace function from PHP function source code).

We had been very upset when found the PCRE checks web page of 1MB near to 0.2 second on our Xeon 3.2 GHz processor (regexes were about 10-20 KB of size). We had realized that we will not be able to serve even 100 simultaneos connects on our SQUID if we will not found the way to optimize it. We also had found that standard UNIX grep works very fast and uses enough of regular expressions syntax to handle our tasks (it takes about 0.001-0.005 second to handle the same document).

Trying to find the understanding of poor performance of PCRE we have moved to mathematical basis of PCRE – finite machines. The theory explains the difference between DFA and NFA approaches used by regular expression engines. To sum up, DFA algorithms are much more faster then NFA, and PCRE is NFA engine. So we had found that we are not able to use PCRE to handle our task and tried to found another tool to complete the module.

We reviewed two concurrent ideas: the first assumed to use grep source codebase to implement regex engine, but this way wasn’t convinient since grep includes global variables and not thread-safe (since utilitity doesn’t need multithreading), the other assumed to use special library and we have found one – RE2, produced by Google when he met the same difficultes as we did. RE2 is C++ library with clear syntax and usage guidelines. The library is thread safe by design, so we have decided to use it in our module.

The implementation of our C-ICAP module with RE2 engine allowed us to achive even faster performance then required with easy to use integration process.

There are great diffrence between web-applications and content resources. Usually, web-applications are much more complex, require a lot of intellegence during analytical investigation and a lot of coding, while the view (presentation) may look moderate or not-so-impressive, from the other hand content resources must attract the audience to monetize content. Keeping this in mind, the work on content project is much more information architecture, SEO and web design rather than coding. Often content projects don’t have complex business processes involved.

The web application is often could be imaged as displayed at picture below:

where there is small part above the water while the most of work is hidden (often called Backend).

Every development team has own preferences regarding the projects to deal with. We have full competence in both areas but historically our major direction is development of web applications, which requires a lot of experience and deep knowledge in the computer science.

There is traditional software development processes could be used at most iterations of web application development. This fact has reasonable explanation – web-application is the business software  by nature and web interface is just interface convinient for customers. The interfaces created for the software applications are often standardized and there are a lot of widget libraries which implement web interfaces in convinent and expected way. You can refer to Sencha or jQuery UI frameworks to get the idea.

From the other case content project are developed in another way. This could be explained in the next way – there is no common  interface suitable for all content projects (customers will not love the project if it will implement some not-so-cool design) so project owners require amazing looking, convinient interface. There are a lot of activities during website design. The first place takes informational architecture and usability design, next – graphical design, next – SEO, and last part of work is coding which is moderate comparing other parts of work.

So, you can see the web site can be like an iceberg where you see only small top while greates masses of ice are below horizon.

GNU/Linux software includes different software packages which cover almost any needs of the business for free. From the other hand, GNU/Linux is very flexible system so it allows to build custom configurations and solutions. One can found that certain task can be solved in couple of ways depending on system administrator quality and personal preferences.

This flexibility in approaches (there are many ways to do it) is not a plus in some cases. Below I will discuss the use of GNU/Linux for routers and firewalls. Telecom vendors had realized the fact that telecommunication hardware appliances must be configured in unified way which is specified by vendor. Even if the appliance is configured by several persons and the configuration changed from time to time, the man who knows how to configure the appliance line will be able to configure certain appliance with no difficulties.

Unfortunately, when certain common purpose GNU/Linux distribution is used to create sophisticated router or firewall there is no single unified way for network configuration provided by vendor (actually no “must-use-way”). There is no single console which handles all the commands and network features, but a lot of isolated services with custom configuration files.

Last years there are different Linux-based solutions created especially for network appliances occured. I want to present one of them, which is deployed in our production environment and function well. Vyatta is based on Debian distribution which means long term support and stable packages. The major features of Vyatta are

1. created for routers and firewall (borders, BRAS, shaping/policing);
2. single ios-like console with command autocompletion;
3. static and dynamic routing support (BGP, OSPF, RIP, RIP2, source-based routing);
4. network statistics export in NetFlow format;
5. L3 VPN (IP-IP, PPtP, OpenVPN);
6. Debian APT support
7. Full featured support of iptables, linux ip/tc;

Vyatta is linux, but very special linux. The main advantage is single, unified approach to configure the system. This means that any system administrator who is familiar with ios-like console and Vyatta will be able to understand the configuration and change it. Extensive configuration documentation allow to find requested information fast. The configuration is applied in transactional approch where administrator must do “commit” or “rollback” operation if he wishes to apply or cancel settings.

Vyatta is free for use, lifecycle is mature and stable. We could recommend it to use in your production environments.