Profiling your software

[a_bertrand]

I thought it could be interesting to continue a discussion with Spudinski about how useful a profiler is, but instead of just having a "yes / no" discussion I thought I would first try to explain what a profiler is, what it could deliver and how you could with the help of a bit more code do some profiling without any external tools.

A code profiler is a tool which check the execution of a code. Could check the time the code / functions takes, or it could check the memory usage (called memory profiler). Both are usually hard to use 3rd party tools, specially if you fire then up the first time. You may find quiet some tools to profile PHP code or MySQL queries. As an example I could point you to: http://xdebug.org/docs/profiler

So what does a speed profiler (not a memory profiler) exactly?

Well basically it will help you to detect what piece of your code takes the most of the time, for example if your code is like:

<?php
$valA=FunctionA();
$valB=FunctionB();
$valC=FunctionC();

echo "$valA $valB $valC";

 

You may want to know which of the 3 function is the slowest, and maybe inside each function you would know which block takes more time etc. Now it's not only important for a function you call 1 time, but think you may have a quick function you call millions of time, then suddenly this function actually is more important for the optimization that a slower function you call only once.

Ideally you would want to know up to which line of code which one is the slowest, but usually profilers ends at function calls so if your script is one big flat thing without functions, then... no luck.

Now without the need of setting up some odd tools, you can directly do some primitive profiling just with PHP and the function microtime. To do so, simply start to store the microtime before the piece of code you want to monitor, and then the microtime after, and by doing the difference of the 2 you know how much time you spent on that piece of code:

 

<?php
$start=microtime(true); // to get it as float
for($i=0;$i < 1000000;i++)
$a="Line $i";
$diff=microtime(true)-$start;
echo "Milisec to comlpete: $diff<br>";

 

Same technique can be applied basically everywhere, and for example I monitor how much time my queries need to complete. You could save this data inside the session or in a special log file, it's up to you. As an example, inside NWE I use the same technique for the included profiler:

[ATTACH=CONFIG]629[/ATTACH]

So how does it help you? The faster a page loads the more page you can serve with a same server, which means the more players you can handle. By discovering which piece of software is slow you are able to try to optimize the right piece of software. Also, you could discover that some query or code is suddenly really slow and it's not just a network connection issue, and could try to investigate what is going on.

[Spudinski]

I never said a profiler is bad.

I use code profiling nearly every day(along with unit testing), in PHP, JavaScript and MySQL(CSS as well, depending on how lazy I am).

It's a very useful tool, but you have to understand the underlaying principles of how your program is interpreted/executed in order to make clean optimizations.

And therewith, SQL queries should also be profiled to look for optimization to be made manually. Even though SQL(incl. MySQL's) parsers does an awesome job of optimizing the queries at runtime, humans are still at the cornerstone of bad practices.

Alain: I'd think it's appropriate to point to tools, and how they work.

For instance, I use a combo of xdebug, Eclipse/Netbeans, MySQL Workbench to perform optimizations.

A good practice is also to use a large to huge dataset when profiling, at least with MySQL.

[a_bertrand]

I didn't took it as if you said it was bad, but I had the opinion you was wondering what a profiler had to do with the engine. So I thought it would make sense to explain to all what profiling is and what it can bring (roughly, as if you go in details... well you could write books on it ;) )

Totally true Spud, without some knowledge of what you are doing, you will not do much. Same on the large dataset otherwise you will not be able to really test MySQL queries. I must admit I don't use much tools for PHP (also because I don't develop all that much on PHP anymore), however I do use JetBrain .NET profilers when I need some help on the .NET side of my work.

So if you have tools / hints for PHP and MySQL profiling please share with us!

[sniko]

Question. Is the Big O Notation a -somewhat- profiler?

Also, this was a good read :)

[a_bertrand]

Big O Notation in my understanding is when a curve function does a "turn" and goes toward some number like 0 or infinite. Nothing to do with a profiler here IMO.