Jeremy Rempel, one of Interloc’s Maximo performance tuning experts, shared his 3-step process with us. In a nutshell:
- Reproduce the problem
- Identify the culprit
- Resolve the problem
1. Reproduce the problem
To reproduce the problem, take into account the network connection, desktop performance, Java customizations and concurrent user count, as these all play into how Maximo is performing. It is equally important that you personally verify that the action is consistently slow.
Work with the local IT group onsite or remote directly into the user’s workstation. If the system is live, ask the user for a detailed list of actions, fields, and applications frequently accessed.
Bottom line is...
...to solve the problem, you need to replicate it.
2. Identify the culprit
Identifying the culprit is probably the most difficult, yet most important, step of this 3-step process. You need to know what to fix before you can fix it. After all, implementing a cluster does not fix a problem with a database query.
Performance issues typically fall into several key areas: client workstation, network, application and database. And because of this, it’s a good thing that there are tools that can help you determine the source of the problem:
- Start with the Interactive Performance Analysis with Maximo Activity Dashboard (MAD) built into Maximo to help identify the problem.
- Move on to the Fiddler Web Developer Proxy or the Charles Web Developing Proxy Application from within the clients environment to track a single request from client to server. This will help to determine how much time was spent in the network versus the server.
mxe.db.logSQL.Timelimit to identify slow database queries.
- Avoid putting logic into
init() methods. Certain processing may be able to move to an async process such as a
- Simplify your screen. When a screen loads, it initializes the Maximo Business Object (MBO), causing all fields and tables to display on the screen. Complex screens and relationships will load slow.
- Simplify your logic. Embedding many, or complex, conditional expressions can slow the screen down. Consider cloning the application to provide different views.
After identifying the source of problem, you'll know where the bottleneck resides and can determine the next actions to take.
Along with the source, it is important to understand and quantify how the system will perform when under load. Load testing lets you simulate a given user load on the system and recording the results. A load test can simulate several hundred users interacting with the various applications, workflow and interactions.
Try HP Loadrunner, JMeter, or Stresstester to help determine performance under load. (For an even more detailed analysis enable verbose garbage collection in Websphere and use the IBM HealthCenter.)
During the load test, monitor the system from different angles, including:
- Operating system
- Application server
Determine if any of the components is near capacity. The load test script prints results of the response time during different loads, letting you determine if and how the performance falls as the system is placed under load.
3. Resolve the problem
Now that you know what the culprits are, it’s time to resolve them.
Solving Server Performance Issues
Server performance issues can be related to either the Maximo Database or the WebSphere Application server. Try these tips to help solve Server Performance Issues.
- Identify specific use cases with users, identify bottlenecks, benchmark, optimize; repeat.
- When >= 25 concurrent users, consider clustering. Separate the Maximo Integration Framework (MIF), the UI, Cron, and Reporting functions to ensure that they do not compete for resources.
- Only increase the Java Virtual Machine (JVM) settings in the User Interface (UI) with proper testing. Increasing heap size can cause longer garbage collection cycles resulting in pauses and delays. JVM heap settings should only be increased with the
gcnodelay property activated.
- Move the secure socket layer (SSL) off of the application server to a dedicated appliance.
- Work with your database adminitrators and use tools to analyze specific queries. Determine how indexing can be used to speed up performance. Rewrite relationships to optimize them.
- Think outside of the box. For example, if the query to the
doclinkstable was using several expensive subqueries to try to figure out
assetuidto the Work Order table and populating it with crossover domain can make the Work Order save up to 5x faster.
- Remove all custom code, test, and rewrite. Turn MBO code into field classes when possible.
- Avoid putting logic into the
save() mbo method, particularly checking one field. Field classes only get called when the data changes, whereas
save is called more regularly.
- Implement the correct level of logging and avoid using
DEBUG in a production environment.
- Use the correct JVM settings for the server. UI servers should be optimized for reduction of GC pauses and integration/cron will be optimized for throughput. Charles or Fiddler can monitor the network and ensure the server is returning responses in a timely fashion, ensuring that there is not a network issues. Ping should be <= 100ms; < 50ms is ideal.
- Turn on verbose garbage collection in a test environment and turn on
mbocount to determine and capture memory leaks.
Solving Network Performance Issues
Try these tips to help solve Network Performance Issues.
- Enable gzip compression and ensure the appropriate caching is setup on the client and kept-alive on the server.
- Charles can ensure that data is being cached and not repeatedly sent with every request.
- Enable the
maxage setting to tell the browser to cache resources.
- Enable gzip on your webserver or on network appliance such as F5.
- Turn gzip off in the
- Use a hardware cache to move the load off of the application server.
- Consider using Charles or Fiddler to simulate slow network/latency.
- Consider changing the following two JVM parameters:
This 3-step process provides you with a high level roadmap of identifying and improving Maximo performance. However, it is important to note that all Maximo installs are different. The data, queries, workflow, customizations and usage are specific to each Maximo install and can create unique issues for each specific install.
For more documentation, please review and implement the best practices in these available white papers from IBM:
Want to work with a Maximo performance expert? Interloc Solutions is an IBM AAA technically accredited, Premier Business Partner. As the IBM Pulse 2012 Tivoli Smarter Planet Award for Asset Management winner, Interloc believes that we are very well positioned to exceed your expectations!