Debug Magento using Zend Z-Ray
Magento sites can be complex and often times suffer poor performance due to complicated business logic and a large volume of database queries. There are many elements that go into the final construction of a rendered page, with so much going on under the hood it can be difficult to quickly back trace issues in a crunch. Z-Ray gives us the necessary insight to guarantee all of our client Magento applications reach their peak performance and stay there with each release.
This gem of a tab is amazing. The tab in the example shows 2,607 function that ran which is typical for a category page with cache off. I am going to show off three debugging use cases based on information in this tab. The screenshots are just raw grabs of the output of the functions tab that Z-Ray gives you with every page request.
Let's start off with the uncached category view example. The examples highlight a typical Magento extension that offers amazing functionality at the cost of performance and a few dollars. The feature list for this extension is amazing, but it has several out of the box caching issues.
We are in the process of fixing a couple of them based on the real-time function that Z-Ray provides. In addition to the caching issues the extension also had poor uncached performance. Debugging this can be difficult without sorting through a ton of xhprof or cachegrind output. Z-Ray trivializes this process, as you will see.
Uncached Category Page View (5000ms)
In this case we have
GoMage_Navigation still showing some sluggishness when the cache is turned off. This is actually an improvement from its original state. Using Z-Ray we were able to fix several of the issues with it. I don't have a screenshot before the fixes, but the initial issue showed 12 second uncached and cached responses because of the issues within this one extension.
Looking at the functions that are referred to in the graph allows you to debug the functions that are slow. Z-Ray gives you the line number and the file path to the function so its very easy to turn your attention to a sore spot in the code. We fixed the helpers that were slow but are still working on the model and blocks.
Improvements in the cached response were significant. Response times were less than 90ms after the fixes, down from 12 seconds. Before the fixes were in place the
GoMage_Navigation was causing the cache to break making every category page as slow as the uncached category view. Using Z-Ray allowed us to focus on the high value fixes, and move on to other issues.
Cached Category Page (86ms)
The Functions tab allows you to filter, sort and find any function that runs. It has a very nice export screen that can be used for profiling reports. A raw output option would be nice so I could use it for graphing without having to text mangle it with sed.
Example: Filtering cache rebuild process for Redis
While I was setting up screenshots for this blog article I ran into two undiagnosed issues on the product page. It really is amazing how fast you can find inconsistent behavior in the caching and the pages when you have Z-Ray to compare with.
Z-Ray really has been filling the real-time function tracing gap for me and has provided very nice screenshots to show the improvements to the clients. It looks great and I feel like its a lot more intuitive to use than xhprof or cachegrind. If it only had a visual callgraph like xhprof, it would be perfect.
These are just a couple examples with this one project. You can imagine all of the bad code that you can easily find using Z-Ray.
Finding what events fired on Magento can be a challenge, there are a few ways to do this already. Z-Ray provides a very nice filtered/sortable results tab for wading through observers that you won't find outside of Alan Storm's Commerce Bug. In this case the functionality is even better than Commerce Bug. The duration column really helps track down slow observers.
The Events Tab
This panel can be very useful in tracking down the events you want that happen right before or after something happens, without having to search for them. I have also used it to confirm my events are firing.
Here again we find Z-Ray overlapping with Commerce Bug. I really like the tree view that Z-Ray has for this section, however it feels a little unfinished compared to the other tabs. It doesn't have an export function, and copy/paste leads to a lot of noise.
I have been able to confirm blocks are loading the correct templates. It's very quick to check here as you are coding an extension. I wish this were searchable/filterable in addition to having the tree structure.
Using Z-Ray on Magento Upgrades and Installations
I used Z-Ray to upgrade an existing Magento site from 184.108.40.206 to 220.127.116.11. This went pretty well, and I wanted to have a little a/b test data from the same server, but I was unable to complete this because of some issues with my vm, or perhaps some sort of bugs that I will be reaching out to Zend to help sort out.
The Magento installer is a bit of a beast, and debugging it can be very difficult. This is due to the installer forwarding you along for errors, or next steps.
Z-Ray allows me to look at transactions from the server without having to view the response. Zend Server allows you to watch the Z-Ray via an interface on the backend so you can see the requests that happen even if you just get forwarded along.
Z-Ray Live shown in Zend Server Interface
View real-time info on all the requests being made to your Web server, including non-browser based requests, such as APIs, Web services, and mobile requests. All the information recorded by Zend Server and displayed in Z-Ray, such as request execution time, events, exceptions, errors, database queries, functions, and more, can be viewed in one centralized location.
If you're doing an upgrade and the SQL fails, having a log of what ran in what order can be very useful and Z-Ray gives you that with an exportable report ready format. Showing you the complete queries.
Z-Ray Live Queries showing the upgrade process
The complete SQL report has all of the queries, including the data that is inserted. This is useful on install of extensions, cronjobs that run, and a few other things as well.
As far as my a/b testing, I tried to install 18.104.22.168 and 22.214.171.124 but ran into a lot of issues with 503 errors, and partial installs. I suspect some VM server settings, but was not able to confirm this before publishing this article. I will be working with Zend Support on this and a couple other questions I have over the next couple weeks and I expect to write more about the proper development VM using Zend Server.
Using Z-Ray Requires Zend Server 8
This is not as bad as it sounds, really. Zend Server installs clean in its own code pool in
/usr/local/zend/ it uses Nginx and lighttpd and is very robust from what I have seen. The install of Zend Server is easy, and the support from Zend is great. I think it's basically inexpensive for the toolset and support.
You can install your normal php stack and configure Nginx to work alongside Zend Server. This is a bit more complex and would require some finesse to work correctly. Using Zend Server does not need to interrupt your normal workflow and server setup, with a little effort you can have them running side by side.
Another upside to using Zend Server is the direct integration with IDEs. You can do a lot with it that I have not covered in this article.
One more plug for Alan Storm and his extension Commerce Bug. This lightweight Magento extension does more than 50% of what Z-Ray does, and it even has features that Z-Ray doesn't have. Commerce Bug has been one of my must have to code Magento tools, and Z-Ray has come to be something that I rely on just as much. If you do not use Commerce Bug already, you are missing out. Thank you again Alan for saving me an uncounted number of hours over the last several years.
What Z-Ray for Magento Needs
The Magento community can add any functionality they want to Z-Ray. There is an extensible API, with good examples and some documentation. Z-Ray is a framework that can work with any application, and already at a great starting spot. Take a look through the Zend Server Extension github for some examples of how this is done.
When you compare Z-Ray against Commerce Bug you can find several things that are missing from the various views. It would be very nice to have insight into collections and the CRUD models. Worse yet the layout tab in Commerce Bug is a must have, and Z-Ray doesn't present all of the missing info. It would also be great to have a layout tab like Commerce Bug does. Although, with the Z-Ray extensibility API, all these missing features should be easy to add to the Z-Ray Magento Extension.
Commerce Bug Layout Tab, including links to rendered layout xml
One of the best parts I have found about Zend is that they are very tangible and happy to get feedback, or give support. They may take a lot of the feedback they receive and add more functionality to the Magento Z-Ray extension and to Z-ray itself. Or we can just add it ourselves.
Try out Z-Ray today. There is a free trial forZend Server and you can even try out a live demo with Magento: http://serverdemo.zend.com/magento/
Over the last 8 months I have taken a real interest into Zend Server 8 because of Z-Ray. This article is a description of some of the things I have debugged using Z-Ray on a current project.
The initial impetus @zeevshttps://twitter.com/zeevs/status/492234652862529536 tweet about Z-Ray. I immediately installed and started to play with it. It has come in handy a few times since then. I hope this article will bring even more Magento developers to Zend Server and Z-Ray, or at least it highlights a few of the key features that Z-Ray offers, and how it may be of use to other Magento developers.
I will be following up with more blog posts with deeper dives into specifics about using Z-Ray. Next week I will be presenting some debugging live on a Zend webinar, I will be showing some typical Magento anti-patterns and how to locate them quickly using Z-Ray. Last but not least I have some plans to integrate some of the functionality that I can't live without into the Z-Ray plug-in.
Take a look through the resources in the following links:
- Zend Server 8: http://www.zend.com/en/products/server
- Try the demo of Z-Ray with a default Magento:http://serverdemo.zend.com/magento/
- Set up documents for Zend Server are found here:http://files.zend.com/help/Zend-Server/content/installation_guide.htm
- Commerce Bug: http://store.pulsestorm.net/products/commerce-bug-2
- Magento: http://magento.com/
- Go Mage Navigation: https://www.gomage.com/gomage-advanced-navigation.html
- Zend Server Extension Github: https://github.com/zend-server-extensions/