So let me just say it up front, “Index the objectClass attribute”. No really, go ahead and do it right now. Now that we have indexed our objectClass attribute, let me tell you  a story about how I learn that whatever you do, just index the objectClass attribute.

Background

I was working on a medium size openldap instance, about 70k records, it’s the user data for a university. We’ve spent the whole day migrating the old openldap setup to a new HA setup with rplication. But at the end of the day we were having problems with really slow queries. A typical “cn=a*” wildcard search takes about 20-30 seconds. Which, honestly, is stupidly slow.

My boss, bless him, decided to dive the logs. Well, technically he didn’t dive the logs, but he did dive into all the output from openldap running on the foreground with the -d option. Which is just as, if not more, tedious. Me, I can go through the most incredibly bad code and still survive, but log files are just not my thing. I just can’t do it especially after a long day. So rode shotgun, googling whenever he found a interesting piece of log output. And providing an extra sets of eyes for anything else.

After some time and experimentation it became clear that openldap is not going to the index for the cn or sn search, but instead bork out with index not found on objectClass and went merrily ahead and doing a full tree search. So we went ahead and add in an index for objectClass, did a slapindex and rerun the queries. The result came back in around 200 miliseconds.

Hind sight

After the event I thought to myself, that maybe I’ve did a WTF and forgot a very basic thing. Surely I the admin guide would have mentioned something like this. So I went through the openldap admin guide again , and then I went to the openldap faq-o-matic, and then I hit google. The only mention that I could find is a guy saying in the openldap newsgroup, mentioning not to remove the objectClass index  in regards to another unrelated indexing issue. ( Can’t seem to dig up the link to that from google groups now ).

I must admit that it is uncommon for anyone to remove to index that was there by default, I really think that there should be some sort of documentation that explain this.

The browser is open sourced and is based on the webkit project which powers browser like Safari. Google apparently have a little FUD going on there with their mention of browser tabs that eats up memory in the comic(cough, firefox, cough).

How good is it? I can’t be sure yet but I’ve downloaded it and using it right now to write this post and I plan to coontinue using it for the coming weeks to see how it performs.

My prediction is it will critical mass as soon as someone comes up with a killer pluggin that lots of people use. My other prediction is that the pluggin will most probly be somekind of google service pluggin ( gmail maybe? Hmm?) , and more probly made my google themselves. We’ll see.

Blogscoped have a very good rundown on the features of Google Chrome.

Speed, reliability and security with an open system is what Google promise, let’s see if they can live up to their big words.

Programming is a creative process. Despite the technical nature of programming codes, the process of coding is involves a lot of creative thinking and problem solving. Paul Graham talks of “Hacker and Painters” and compares the work of us code ninjas to the creative process of painting. And like all other creative task you require a certain amount of “flow” to get going with it. Many programmer talks about “getting in the zone” where once they get their flow going, productivity soars and work that has taken a few days to figure out suddenly seems easy and is done in a matter hours.

Getting in the zone

What do you do But how true is “the zone”?   Is this a guise in which we programmers procrastinate under? Do we truly belief that “just one more turn” of civilization, or that “one more level” deeper in nethack is really going to make us have a clearer head to code with?

I myself is a believer in the zone. I code my best late at night blasting Metallica through my headphones. And when I was stuck, an hour or two at gaming would be the answers. It worked well enough when I was doing it. But now I that I got a family, staying up late is no longer an option, my wife have to wake up early to go to work and the computer is in the bedroom. My new job also means that I can’t keep the hours that I used to. So now I have a small window of time after dinner and before bedtime that I have to code. I HAD to do it then because I would never have the time to do it at any other time. But still the code flow and work gets done. There maybe a few times when I have “coders block” and can’t think of a solution. I would then wrap up the day’s work and continue tomorrow. But then again even in the old days of midnight coding sessions I would also come across some speedbumps every now and then.

Another example is when dateline is near. Even if you never had a change of lifestyle that requires you to suddenly change your coding schedule, almost everyone have at one point or another be on a tight dateline. And no matter if you are in the zone or not. If the dateline is tomorrow, you still get stuff done at crunch time. So why does suddenly the zone doesn’t matter. Is it an illusion that we put up to justify our laid back approach and procrastination?

Fiction, myth or excuse?

Of course a quite environment that allows you to work without interruptions are essential for doing out work. So is a good tool for your job, for example a big, quality monitor for a designer to work on or a good IDE for the coders. But how do we justify arcade cabinets or networked starcraft sessions during the day or a trip to The Dark Knight Showing during work time.

Do you belive in getting in the zone? Do you have to do anything special to get in the zone? Do you think that it is about time that we own up to our procrastination and stop using the excuse of trying to get into the zone to avoid work.

Pictures from davestfu

Updates

7th Aug: I’ve a a few reddit traffic, courtesy of this,that saying that I am “claiming false dichotomy”. I’m not claiming any dichotomy at all. In fact the opposite, I’m saying that may be we could own up to our own lack of motivation instead of hiding behind the excuse of not in the zone. I still believe you have to be in a good environment to be able to code and a quite place to work with no distraction is a must and that will contribute a lot to being able to be in the zone. But sometimes work is work and it is tedious or boring and we just have to do it and if you set up conditions that are conducive to working then the flow will come.

Sorry for not being able to convey that message clearly.

Code readability is paramount in an environment where you are not the only one reading and writing the code. And even if you are the only developer of your system, I know more than a few coders that read a code that they wrote last week and thought “What the hell was I trying to do with this code?!” So, give your fellow coders a break and follow these 6 tips to make your code readable and make your fellow coders love you.

1.    Consistency

Consistency is the number one rule when writing readable code. It doesn’t matter if you but your braces in the same like as the if statement like this:

1
2
3

if (x==0){
    //do stuff
}

or if it’s on the next line, like this :

1
2
3
4

if (x==0)
{
    //do stuff
}

Just pick one and stick with it. Follow the C++ coding standards or follow the gnu coding standards or follow the java coding standards or make your own standards, it’s up to you, but chooses one and use it everywhere. It’s very annoying to see different kind of coding style in the same project, or worse, the same source file. Create a “Coding Standards.txt” and put it on the top directory of your source code so that all the coders can see that you follow, and expect them to follow, a coding guideline.

2.    Indenting

Indenting is one of the most important aspect of code readability. Indenting allows readers to have a general overview of the flow of the code. And it also provides a visible cue the code’s structure. For me personally, proper(and more importantly consistent) indenting is one of the biggest factor that contributes to code readability. The general rule is that all logic block should be indented, and any logic that branch from the same point should be indented at the same level.

3.    Comments

Many people argue that a lot of comments makes a code readable. But for me the comments should be used sparingly. A well written code should tell its readers what it is doing just by looking at it. Comments should be used however to indicate why a certain part of the code does what it does. For example:

1
2
3
4
5
6
7
8
9
10
11
12

//example 1
i=0;//initialize i
while (i<1000000){//while i is less then 100
	i++;//increment i
}
 
//example 2
//this is part of out test suite to stress the CPU 
i=0;
while (i<1000000){//an arbitaryly long number
	i++;
}

The first example while having more comments is pretty much useless since we already know it is initializing the variable and we already know that it is comparing against a million and we already know that is is incrementing the variable from the code. So the comments add nothing to the understanding of the code. We want to know why it is doing the loop.

The second example solves this by providing a reason for the code to be there and also provide useful information like why the number 1 million is chosen. If you see the code in a production environment then we KNOW that we can remove it because we know what it does. But if we see the first example we might be hesitant to remove it, why is the code here? Does the program require time to finish its task before running the next line and the previous developer never heard of sleep? What is the significant of the number 1 million. Is that the time to live for some unknown cache that we are waiting to expire?

Let the code tells most of the story and what the code can’t tell, you comment.

4.    Variable names

Following my stand that code should obvious and tell its readers what it is doing I believe in descriptive variable naming. I really don’t mind a long variable but numberOfCommentsFromUsersThatAreRegistered is really tiresome to type and read and more likely than not will cause horizontal scrolling on the part of the reader. The basic rule is to be as descriptive as possible but to drop grammatical articles from the variable name. So numberOfCommentsFromUsersThatAreRegistered would become something like registeredUsersCommnetNum. Short forms should be avoided unless it is a really common convention like the prefix is for any function that checks if something is a certain type/condition etc. or the numOf prefix. In strongly typed language like java or C it is common to prefix the name of the variable with the data type. For example intCommentCount is an integer and strVehicleName is a string and so on. Other stuff like underscore use and capitalization shouldn’t matter much but, as per item 1 in this list, pick a style and stick with it.

5.    Minimize horizontal scrolling

When writing code make some effort to minimize horizontal scrolling on your code. The typical screen size is 80 columns and you should try to make sure your code fits this. Horizontal scrolling make it harder to read because of the effort required to actually even see the code. On modern IDE the person reading the code would need to move the mouse to the bottom of the screen and click on the scrollbar. Which is bad enough but then if he needs to read something above or below the code he probably needs to scroll back to see it. On emacs it’s just plain annoying to scroll horizontally. I know there is word wrapping but that always make the code looks bad and ‘wrong’. Most codes can be broken into multiple lines or multiple instructions while keeping the same behaviour.

6.    Follow the gnu convention of doing one thing and doing it right

This apply to writing functions. I always cringe when I see a function called “process” or something like that. Never mind that is it is an ambiguous name to start with, it also implies that the function will do a lot things. For example it might filter the input, then validate it, then store it in the database. It would be much better to create one function to filter the data, another to validate it and another to store it into the database. It is easier to debug that way since we can isolate the processes in the function and also easier to extend because you can, for example, change how the program validate the data without messing with the database code. And more importantly and to the point of this article it is easier to read smaller chunks of code that you know is doing a single thing that to plow through a whole page of code that could be doing either of the three things mention.

The End

Now go write your code and make your fellow coders love you.

A selfsigned cert is a digital certificate that is created by using a certification authority certificate that is issued by yourself. This is usefull for backend communications between servers that need the protection of a SSL tunnl but don’t need the ‘assurance’ provided by a regonized and trusted third party certification authority (and the charges that associated with obtaining one).

This is usefull in cases where communications between appserver and the data layer (typically RDMBS) needs to be protected with SSL. You just need to create a selfsigned cert and have your applications trust your own CA cert. You can then encrypt all your backened communications and only use certs obtained from trusted CA like, for example, Verisign where you want custumer confidence. And if your custom application need access to webservices you can make the application trust your own CA cert. There are many things other that you can do that doesn’t require a commercial SSL cert.

Approach

The people at Riseup Lab have already made a great guide on selfsigned cert. I have follow the steps that has been outlined there and created a shell script to create the selfsigned cert without going through the process of doing the work manually.

Basically, I take all the steps outline in the guide, put it in a shell script, do some externalizing on the parameter to make it a little configurable, and tested it abit. I use the term tested loosely here since all I did was try it in different directory and with some differences in the settings. It has some basic error checking but for the most part it’s not very robust and stills expects you to put in the correct stuff for it to work.

Running it

createSelfSignedCert.sh

Just download it above and run the supplied shell script and it will generate everything that you need to create the selfsigned cert. If for whatever reason you want to reset everything. Cleaning up the database of old certs and start fresh and create a new CA cert, just run the script with an extra “fromscratch” parameter, this will clean everything and start from scratch.

1

./createSelfSignedCert.sh fromscratch

Your thoughts

I would very much like to hear what you have to say about this script and how it can be improved. Please leave a comment if you happen to try it out.

Ugh.. We don't want this ugly html tags showing

See the <p> tags? We don’t want that. We want to provide the submiters the ability to have some html formating power and yet we don’t want to allow them the use of fancy XSS or javascript on the site. Keeping with the opensource spirit we will go ahead and use another freely available component to filter our input. It is called HTML Purifier.

Getting HTML Purifier

Head on over to HTML Purifier download page and download it. There is a lite download that strips out all the extra documentation etc. from the package. That really is all that we need but the documentation does comes in handy sometimes. So I recommend the full package.

Installing HTML Purifier

We need to put the libraries somewhere and, like we did with the tinymce library in the previous article in the series, the lib directory in your phpLD installation is perfect for this. Just copy the whole library directory into the lib directory and rename it to something more descriptive, ‘htmlpurifier’ will do.

This is what your lib directory will look like

Configure

[The few paragraphs below are just some of my thoughts about why I do things a certain way. They are kindda rant-ish. So you can just skip to the "Method 1: Handle it at code level" section if you want]

Ok now we need to configure it so that phpLD will know how to use it. There two way to go about this.

One is to do this during the process of getting the data from the database. Purify the data as we get the data then pass it to the template engine. This is usually the way I would prefer to do it. But the way phpld get links info is to give it to a variable straight from the resultset of a sql query. So there’s no way to intercept it, so we need to take the $links variable and travers through the resultset and filter it after it is assigned. Seems like a hack to me.

The other way to to send the data as is and let the template system handle the purifying. This involves writing a smarty plugin. Which is a better solution, in this case in my oppinion, but if you change theme then you have to remember to update the theme.

I’m leaning towards the first option really because I feel that this is a logic code rather than a UI code. And, following the MVC convention, should be handle by the code before handing it over to the template system. Either way, I’m really divided with this and I’ll just show you both ways(I’ll show the template method in the next article) and you can try out the ones that you thing fits your purpose.

Method 1: Handle it at code level.

Include the libraries

First you need to include the htmlpurifier include file in init.php. You need to add an extra include right after all the includes for the smarty and other libs includes.

You need to insert your include at the end of the other libs includes. Around line 52

After line 52, add the following line

52

require_once 'libs/htmlpurifier/HTMLPurifier.auto.php';

Calling the purifier

Now that the includes are done with. We need to go and call the purifier on out data. Go to index.php, near the bottom, right before we assign the variables to the template and call it.

Index.php.Just before assigning it to the template. Around line 178.

Here we need to insert to following.

180
181
182
183
184
185
186

$purifier = new HTMLPurifier();
if ( (isset($links)) && (!empty($links)) ){
	foreach ($links as &$link){
		$link["DESCRIPTION"]=$purifier->purify($link["DESCRIPTION"]);
 
	}
}

Disabling the smarty filtering

One more thing that we need to do is to disable the filtering that is done by smarty. We need to go to link.tpl. And remove the escape modifier on the smarty variable {$link.DESCRIPTION}

Remove the escape modifier on $link.DESCRIPTION. Around line 11.

This is of course will be a little different depending on your template.But the important thing is that you remove the escape modifier from $link.DESCRIPTION.

11

{$link.DESCRIPTION|trim}<br />

And you’re done.

Yay. Now we have bold text.

Other considerations.

Remember that htmlpurifier is only PHP5 compatible. There is an old php4 version that you can use if you are still using php4. (You really should move to PHP5 :D).

You can also specify exactly which HTML tags that you want to allow and which ones you want to disable. But for me I think that the default that comes with htmlpurifier is good enough for me. You can consult the documentation on how to configure these.

The manual also have something to say about the loaders. If you you are using opcode cache like APC and the likes. The htmlpurifier autoload will have some problems with that and you need to spicify another include files that doesn’t use the autoload functionality. The manual also touches on performance issues. If you are concerned with that then you definitely should read the manual.

Summary

htmlpurifier is a good way to allow more flexibility to your submitters. There are many things that you do with it. For example letting premium listing have access to formatting that allow them to make their listing stand out more. It’s all up to you how you want to implement this.

Let’s get started then.

The 2 main rich text editor right now is TinyMCE and FCKEditor.Both are good but personally I prefer TinyMCE, so we’ll go with that. First download a copy of tinyMCE from tinyMCE download page. The compressor php version is better because it is compressed so it downloads faster. But if you already have gz compression enabled on your website then it will cause problems and you need to use the plain vanilla version. The compressor php version is actually downloaded as an extra to tinyMCE, so if you want to use the compressed version you have to download both the normal version plus the compressor PHP version.

Now unzip the archive. The most logical location for the tinyMCE files are in the lib folder. So create a new tinyMCE folder and put everything in there. You directory should look more or less like this.

tinyMCE is now ready to be used. So we need to add the includes to the .js files.

A good place to put the includes is right after all the other includes in the admin template. After line 31.

1
2
3

<script src="files/admin.js" type="text/javascript"></script> <script src="files/ie5.js" type="text/javascript"></script>
<script src="files/XulMenu.js" type="text/javascript"></script> <script src="files/browser.js" type="text/javascript"></script>
<script src="files/tooltip.js" type="text/javascript"></script> <script src="files/pop-list.js" type="text/javascript"></script>

I am using the compressor PHP version so this is my include:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

{literal}
<!--tinymce-->
<script src="/libs/tinymce/tiny_mce_gzip.js" type="text/javascript"></script> <script type="text/javascript"><!--
tinyMCE_GZ.init({
	plugins : 'style,layer,table,save,advhr,advimage,advlink,emotions,iespell,insertdatetime,preview,media,'+
        'searchreplace,print,contextmenu,paste,directionality,fullscreen,noneditable,visualchars,nonbreaking,xhtmlxtras',
	themes : 'simple,advanced',
	languages : 'en',
	disk_cache : true,
	debug : false
});
// --></script>
<script type="text/javascript"><!--
tinyMCE.init({
	theme : "advanced",
	mode : "textareas",
	plugins : "contextmenu, searchreplace",
        theme_advanced_toolbar_location : "top",
        convert_urls : false,
	theme_advanced_buttons3_add : "search,replace"
 
});
// --></script>
{/literal}

The first <script> tag is the gzip loader used to load the comppresed tinymce files. This is not needed if you are not using the compressed version of tinyMCE. The second <script> tag with the tinyMCE.init function is the important part. Here you can specify the options you want for your rich text editor.The option in it’s most basic form is:

1
2
3

tinyMCE.init({
mode : "textareas"
});

Which will give you the output like this:

You could customize it some more according to the options at the tinyMCE wiki. For me I finally settled for the above options which gives me this out put

You could, if you want to, only enable the rich text editor on only some of the textareas that are identified by a certain id. But personally I think that the few textareas that phpld has (one on submit page, one on the admin approve page, and one on the add/edit category page) are the ones that needs the rich text editor.

You could also put the rich text editor on the submit page by adding the tinyMCE includes on the header.tpl file in the template folder. It will allow users to submit site descriptions with the rich text editor. But I really would not do that untill we do some tags filtering on the output (which I will show how on the next article in the series) to prevent any abuse of the rich text editor.

I’ve been wanting to get my hands on a toy project to play with when recently I’ve stumble upon phpLD. phpLD is a popular directory script. The version 2 of the product is the forgotten brother of the more advanced and paid version of phpld version 3. phpLD version 2 is still widely used by a lot of people. Mainly because it’s free and it works. It may not have all the nifty features and mods of the v3 line. But it is a solid directory software albeit a little barebone.

It is a good oppurtunity for my to sharpen my rusty coding skills with this project because it have a solid code base that I can use as a jump off point. so that I don’t need to spend a lot of time coding basic framework stuff. And secondly because there are a lot of ideas already implemented in phpLD3, if I ever ran out of idea on what to do with it I can easily look at a feature that is present in v3 but not implemented in v2 and try to do my own implementation of it. Finaly, because phpld2 is still widely used, any improvement (if I ever make any) will benefit a lot of people and give me a warm fuzzy feeling :D.

For the past couple of days I ‘ve been breaking in the new wordpress 2.6. And all around I am impressed by the improvements of this version. But before I go to any details. The most obvious questions is of course “Should I upgrade?”. My answer to this is a big YES!. This main reason for this is:

1. No compatibility issue
   • Unlike the 2.3 -> 2.5 upgrade. This upgrade doesn’t change the theme and plugins architecture so you don’t have to worry about the plugins breaking or the theme breaking. If it worked in 2.5, it will work in 2.6.
2. Security
   • Along with the many bug fixes, wordpress now also adds SSL security as a core functionality. You can now force SSL connection to yout wp-admin. This might not be much but it is a step in the right direction security wise.

Basicly it’s an improvement all around and you have no reason not to upgrade.

Plugin Handling

Plugin handling have received a big overhaul in terms of usability. You now can mass activate or deactivate plugins, a featured that have been requested many times. This is very helpful when upgrading or debugging when you need to deactivate a lot of pluggins at once. Considering that a typical site will have around 20 pluggins this is a really big time saver.

Another thing that I really like about the new pluggin page is the pluggins are grouped according to Active plugins, inactive plugins and recently active pluggins. This is really useful when you need to reactivate those pluggins that you deactivated earlier. Instead of digging through the long list of deactivated pluggins, you can just browse the recently active group and clearly see which of the plugins that has been deactivated recently. Furthermore shift click works as you would expect and select a range of check box saving you even more time instead of clicking on each checkbox.

Wordpress recently active pluggin

You could see a little similarity  between the functionality of the the new wordpress pluggins page with the PlugInstaller pluggin. And you will see later that this is a somewhat recurring theme. The developer has taken a useful functionality that a plugin provides, improve it and integrate it into the core of the system.

Post revision

Owh… I just love this. Your post will now have revisions. And what’s more you can pick any to revision and put it side by side to get a diff between the two versions.

Wordpress post revision - showing a diff of 2 revision

Very handy. I personally write stuff in bits and chunks,  especially if I’m doing a how to. I’ll write down the major steps. And put in details and revise old sections as I go through the process of creating the how-to. Sometimes Stuff get lost because I deleted a section that I think I don’t need or something like that. But now with post revision it’s like you have a SVN for your post inside wordpress. Very cool.

Improved media uploader

I don’t use the media uploader much so I can’t say how improved it is now but one thing that I really love is the image caption functionality( as seen in this post). I used to use a image caption pluggin to do this. And I love it that they included it in wordpress.

Press this

This is another useful feature. You could drag a bookmarklet to your bookmark toolbar and drag anything onto it and wordpress will add that item to a ‘basket’ so that you can insert it into your post. This works with anything from links to images and even youtube videos and flickr image. This is super useful while doing research for a post.

Theme preview

Now when you click on a theme in the design tab, wordpress will provide a theme preview ala wordpress.com theme preview. Wuuhooo…

Theme Preview

I absolutely love this. When I started to move from wordpress.com hosted blog to self hosted blogs I was surprise to find out that the theme preview I loved so much is not actually in wordpress. I search forever for plugins that can do this to no avail. Thanks wordpress team.

Google gears

With google gears what the wordpress team is storing all the images on your site and loading it straight from your harddrive stored cache. This makes surfing very fast especially in themes with lots of images. Google gears is very powerful  and it’ll be interesting to see what else they will do with google gears and wordpress.

Summary

No backwards compatibility issues with pluggins and themes. More security. More features. Really, just update already. Good job wordpress theme for this release.

If you haven’t heard wordpress 2.6 is released today. What a you waiting for? Go get them now.