Author Archives: Shagul

  • -

OSGi Adoption and Liferay

Category : Development , Liferay , OSGi

Where Do We Start ?

The goal of this article is to provide a perspective of how relevant the adoption of OSGi in Liferay is to a business executive. Is it really worth the effort to learn OSGi? There are many ways to develop an application. The are numerous frameworks in various languages such as Java, JavaScript, Ruby, PHP, Python etc. If you are a firm that does not deal with Java technologies, OSGi is not for you. If you have a Liferay implementation or are considering one, continue reading.

A Little Bit of History

OSGi has been around since 1999. Liferay has invested more than 5 years in the OSGi technology. The last two years probably were the most intense of all when Liferay started migrating most of the core portlets and services. Until recently most of the Liferay developer community could have easily ignored OSGi and continued to develop plugins the old way. The recommended approach going forward is to use the OSGi bundle, though a legacy deployment may still be supported.

Why OSGi in Liferay?

This is what Ray Augé had to say over the years. Ray has been leading this effort and is the key player behind this implementation.

“Liferay is a large, complex application which implements its own proprietary plugin mechanisms. While Liferay has managed well enough dealing with those characteristics over its history, it’s reached a point on several fronts where these are becoming a burden which seem to bleed into every aspect of Liferay: development, support, sales, marketing, etc.” – Ray Augé October 11, 2012

“Liferay is a complex beast with an intricate network of features, so many features in fact that they occasionally have such indistinct lines such as finding the where one feature ends and another begins can be difficult…The number of benefits is almost too great to list. However, one of the greatest advantages can’t be discussed enough: Modularity.” – Ray Augé Feb 4, 2013

The primary reason Liferay adopted OSGi is to easily manage Liferay as a platform. It is to make things easier for core Liferay developers. The key benefit is the modularity of the OSGi platform. OSGi allows the end user to easily add/remove/enhance services and offerings dynamically. The hardware industry has been following a very modular approach that allows us to add and remove components easily. There is a software piece to it as well which recognizes those dynamic components. Not all software is developed in such a way. Also it should not be the responsibility of the software but rather the platform which should enable such modularity. A capable platform ensures that we don’t have to reinvent the wheel on every piece of code. This is one of the promises of the OSGi platform. If you can tell what your piece of code does and someone can tell what their piece of code needs, the platform can match that up for you while you are still up and running. The OSGi platform does offer various other benefits that you can read up on.

How Relevant is OSGi in Current Architecture?

If you are a business that is using Liferay or looking to use Liferay, it is important that the team invest the time to learn the basic concepts of OSGi. OSGi has a very dedicated group of members who have given all that they have to keep it more relevant. It is very likely that Liferay will continue to use OSGi for at least another 5 years (estimate based on what it took to adopt the OSGi technology). So the time spent on learning OSGi while using Liferay is not a waste.

OSGi provides the concept of µservices within a single JVM. Liferay primarily relies on this feature. In order to stay relevant with the modern cloud architecture and distributed services, there are various OSGi initiatives such as Amdatu that embraces cloud computing.

You could entirely develop a full fledged application using OSGi. Just like you would with Spring Boot and Angular JS or any other JavaScript, PHP, Ruby, or Python frameworks. We could put it this way. Instead of saying, “How relevant is OSGi?,” we can say that OSGi is trying to be relevant by making use of all the new technologies. One thing that the community may lack is the funding and hype similar to the some of the platforms.

Is OSGi the Only Way to Build Modular Applications?

OSGi is probably the only best way to build modular and dynamic application using Java within a single Java Virtual Machine (JVM). The key thing to note here is the single JVM. The way software architecture is evolving indicates that the applications are built in a more tolerant way so that you could remove a server and add it back in a matter a few seconds to a minute or two.

The concept of changing class or implementation within a JVM is not relevant if you are already a shop that knows how to build and deploy your application to an elastic cloud. At that level you are elastically scaling virtual machines in a more tolerant way.

Spring Boot along with PaaS providers such as Pivotal Cloud Foundry and Heroku are an alternative for those developing using Java. OSGi Enroute is trying to provide similar capabilities where you can bundle up your app as a jar file and run it anywhere. Along with the pipelines offered by some of the PaaS providers, nowadays it is as simple as committing the code and the rest is taken care for you.

If you are familiar with some of the javascript frameworks, they do a whole lot of things without having to worry about the class loading issues. In fact, Liferay themselves are working on a similar PaaS called WeDeploy. As of now WeDeploy is in Alpha. Liferay’s interest in providing such as platform clearly indicates the effort to stay relevant and diversify the risks.

It all depends on what you are looking for. If you are a platform or a tool builder, it makes a lot of sense to use frameworks like OSGi. If you are a already running tolerant applications in the cloud, the modularity offered by OSGi is something that definitely does not concern you.

A look at OSGi Adoption

Eclipse IDE is one of the most successful adopters of OSGi that a Java developer comes in contact with on a regular basis. Spring tried to support OSGi but later dropped Spring DM due to the complexity. Glassfish abopted OSGi but later the project was discontinued by Oracle. Liferay has taken the major step of adoption and successfully launched the major version. There is still a lot of work to do within Liferay, but Liferay does provide good support for those developing against it. OSGi has moved further on with OSGi enroute and various cloud computing offerings. Liferay’s primary focus in OSGi adoption is the capabilities within a single JVM. In my opinion, by doing so Liferay has committed itself and has become a major player in OSGi for web applications. A continued success and user adoption within the Liferay community could very well provide the oxygen that OSGi needs in the web application platform.

Liferay has done the toughest part of OSGi adoption in its platform. For the end users Liferay provides various utilities to interact easily with the OSGi service trackers etc. It is sufficient to just understand the basics of OSGi to develop plugins for the Liferay platform. Developers could deep dive into OSGi as needed while working with Liferay. This is in a way similar to how developers using Liferay interact with services without having to master Spring or Hibernate.

Bndtools and various others OSGi frameworks such Apache Felix , Amdatu etc., help it make easy for the developers. There is still a lot of activity primarily supported by the OSGi Alliance members which keeps the community strong even after 17 years.

Conclusion

There are many ways to develop rich applications. If you are a shop that has invested in Liferay, then getting up to speed with OSGi will make your job easier. If you are not a shop that is invested in Liferay or Java, you could live without ever knowing what OSGi is. The one thing that OSGi lacks is the hype and support from the extended Java community. As Liferay is trying to be more than a portal, as a business you need to think beyond any programming language or a platform and evaluate what is best for you. Technology changes so fast. Instead of adopting technology for the sake of it, you need to adopt it to solve your and your customers’ need.


  • -

Marketing, Did You Get Sold?

Category : Marketing and Sales

I have been trying to understand the difference between marketing and selling. I am especially interested in how this works in the IT industry. Surely, we all can relate to various things that we don’t need or want but end up buying throughout our life.
American Marketing Association (AMA) defines marketing as:
“Marketing is the activity, set of institutions, and processes for creating, communicating, delivering, and exchanging offerings that have value for customers, clients, partners, and society at large.” – American Marketing Association
This is how one of the former professor at Harvard defined selling:
“Selling concerns itself with the tricks and techniques of getting people to exchange their cash for your product. It is not concerned with the values that the exchange is all about.” – Theodore Levitt, Former Professor at Harvard Business School
Quite often we can relate the success or failure of a project to the difference between marketing and selling. We will review some of the pitfalls so that you can spot issues early on. At the end of the day, the buyer needs to make sure that they are getting value and satisfaction for their money.

Pitfalls

1. Just Launched a Site With Millions Of Users

If you hear someone saying that they just launched a site that will serve millions of users, you would imagine that it is the next Facebook. It may very well be. If it is and you are looking to build a similar system, you are in the right hands. In certain cases it may not be. What your million user need may be very different than what the system that was built delivers.

Say you have a need to build a site where millions of users may log in, search, and buy products. The need for this site is very different than the need for a site like health insurance or auto insurance websites. Both the sites may very well have the same number of users but the activity level and need  is very different. In the case of health insurance and auto insurance websites, once you set-up auto-pay for the next six months or a year, you may hardly login unless there is a claim or want to add another dependent. In case of the insurance industry, they may have the need to process and generate documents either on demand or offline which may be more taxing than generating an simple invoice.

Most IT executives are experienced enough to ask the right questions about concurrent users, active users, daily logins, hourly logins, peak, number of transactions etc. But there may be some who may just go with their impulse and buy a solution only to later realize that this is not what they wanted. It would be a few million dollars too late.

2. Delivered Solutions To A Major Company

We lend our ears as soon as we hear some of the major companies names. If one of the Fortune 500 companies adopted this technology, solution, or chose a vendor then they are the right ones for me. It would be very simple if that were the case. A vendor may very well be the preferred services provider for a company and deliver all the IT needs. This is great news. In some instances this is not the case. A vendor might have delivered a very small piece of a solution to a major company and use their name to win something that they cannot deliver. Asking the right questions is the key. The questions that are relevant to your project may be different than the ones that are relevant to others. Instead of going behind the big names, look to see if the vendor has what it takes to deliver what you need.

3. Look At Our Awesome Testimonials

basis2_testimonial

This is an awesome testimonial and any city that needs a new billing system for their city could consider looking into basis2. The one thing that this does not tell is how even basis2, the fifth attempt, was eventually deemed as a failure after a year. After a total of $49 million dollars and five attempts, the basis2 system sent some residential customers $331K utility bill compared to $97 for the previous month.

basis2_sample_bills_customer_received

The company that sold the solution was able to capitalize on a failed project. For those who are interested, you can read the reports from the city at the below links.

City of Philadelphia Chooses Basis2

City of Philadelphia Audit Report A Year Later

Things to Consider

1.If It Works, Keep It

How many times have you heard something like, “If it is working well for you, keep it?” Typically what you would hear from a salesman is you need everything that they sell. Quite often the commission is not delivered at the end of the delivery but much sooner. Many may know very well as to what happened with the Wells Fargo fallout recently and the pressure of sales target.

2. Never Shop Hungry

We all either know this or acknowledge the saying, “Never go shopping when you are hungry.” If you are in a desperate situation, it is likely that you will buy anything that you believe will satisfy the immediate need. A more proactive solution is to continuously assess what you have and plan for the future.

3. Don’t Feel Rushed

If you hear someone say that, “if you don’t decide now, it won’t be available later,” it means the seller is more in need than the buyer. If you like to be in control of what you are buying, never let external factors take that control away. The more I have taken the opportunity to think through what I am getting without rushing, the more satisfied I am. The key here is the satisfaction, even if you end up with something else later on. Let us not confuse due diligence with procrastination.

4. Trusted Partner

The difference between a trusted partner and a con man is as that of an anchor verses a fishing rod. One helps you from drifting away and the other one attracts you with a bait. If you find a trusted partner, continue to work with them.

 

Conclusion

There is a lot that can be written on this topic. Success or failure of a project starts with someone getting sold. If you don’t have a plan for your money, there are people who definitely have plans for them. We learn a lot through failed attempts. It is a much smarter thing to learn from others failed attempts than to insist on failing yourself.


  • -

Application Performance, A Never Ending Journey

Category : Development

Introduction

Application performance has always been a concern even as hardware and memory got cheaper over the past decades. Does it really matter when you can containerize pretty much everything and even run a container per user when they are online? The answer to that is yes. Dynatrace, Newrelic and other Application Performance Monitoring providers all have those markets covered as well. So, where do we start?

Tools

As the saying goes, a good craftsman never blames his tool. You could look at this in a couple of different ways. You make it work with what you have leveraging your expertise. You choose the right tool for the work when you have the option. Once the tool is chosen, complaining about it without the proper course or change of course of action only reveals the poor craftsmanship.

If all that you have is a rock, you could still make a spear out of it. It does not mean you ignore all the inventions and discoveries that came afterwards and get stuck between a rock and a hard place.

Design

No one has unlimited access to resources. Even if you have, it does not guarantee the performance. As explained by Amdahl’s law, the lowest performing link would limit your ability to achieve the output you want by merely scaling. It is just like trying to use a flour sieve to fill a container with water. Design your application for the task it is intended for.

As an example, if your application needs to process a lot of documents, parse them and index them, it may be better to scale that aspect of it without affecting the performance of the system. If you are buying a platform, see if the platform can easily support the scaling as shown below. If not, you need to plan for it with the tool that you have. Search engines themselves are capable of parsing, indexing and storing the documents.

Search Optimized Architecture

Quite often a system may support an external search engine but may not support externalization of document parsing. Depending on your use case, this may or may not be an issue.

Session vs Cache vs In memory Systems

Session was a great thing once upon a time. Everyone was excited about storing user interactions, data etc on a per user basis and replicating that across servers to provide a seamless failover. This architecture requires that you scale your servers vertically. Just like air fills the container, work expands to fill the time, the objects soon filled the memory. The advent of stateless architecture changed the way we viewed user interactions.

I have seen implementations where developers stored common datasets in a per user session. The data that was stored quickly filled the memory as more users logged in. A cache that can be shared across user sessions is a better option since most of that data was a read-only data. A lot of the systems are optimized at various levels. A database has its own caching mechanism. A second-level cache is supported by various systems. Some frameworks offer a third-level caching of complex objects. So, where do you start and stop? A one-size fits all may not be the option where the problems are unique.

I used the term In-memory systems to include a wide range of systems out there. It could be an in-memory database, search engine or any system that can be used to easily store and access data and has a persistent store from which you can rebuild the memory. This could be  a very good alternative to the other two means.

Upfront vs Continuous Process

Sometimes you have the luxury to continuously improve your system. If you have that luxury, that works very well with the Return On Investment (ROI). You could measure every optimization that you perform and keep improving.

You may not have that luxury sometimes. Your application needs to support certain use cases and a certain number of people. If it fails to do so on day one, it may fail to gain the trust of the users.

Design diligently. Perform the benchmark via load testing. The benchmark is a little trickier because you cannot expect the users to follow your load test use cases.

 

Performance Tuning

What about all the performance tuning, properties tweaking, environment settings adjustments etc? All these are useful but never substitute for a poorly designed system. You use the performance tuning as a checklist for the system that you are running. If your system was designed for a certain altitude, a mere tuning may not take you to the next height.

 

Conclusion

Don’t blame the tool. Use the right tool if you can find one. Don’t blame the design of a framework. Understand how it is designed and see if that is the one for you. If you are a team of experts and you know what you are getting into, you could make it work. If you are not a team of experts, find an expert.. Do the due diligence. If you would be happy to claim the success, don’t be afraid to take the blame and make it right.