Category: "Business"

Technology for the OSS DSS Study Guide

'Tis been longer than intended, but we finally have the technology, time and resources to continue with our Open Source Solutions Decision Support System Study Guide (OSS DSS SG).

First, I want to thank SQLstream for allowing us to use SQLstream as a part of our solution. As mentioned in our "First DSS Study Guide" post, we were hoping to add a real-time component to our DSS. SQLstream is not open source, and not readily available for download. It is however, a co-founder and core contributer to the open source Eigenbase Project, and has incorporated Eigenbase technology into its product. So, what is SQLstream? To quote their web site, "SQLstream enables executives to make strategic decisions based on current data, in flight, from multiple, diverse sources". And that is why we are so interested in having SQLstream as a part of our DSS technology stack: to have the capability to capture and manipulate data as it is being generated.

Today, there are two very important classes of technologies that should belong to any DSS: data warehousing (DW) and business intelligence (BI). What actually comprises these technologies is still a matter of debate. To me, they are quite interrelated and provide the following capabilities.

  • The means of getting data from one or more sources to one or more target storage & analysis systems. Regardless of the details for the source(s) and the target(s), the traditional means in data warehousing is Extract from the source(s), Transform for consistency & correctness, and Load into the target(s), that is, ETL. Other means, such as using data services within a services oriented architecture (SOA) either using provider-consumer contracts & Web Service Definition Language (WSDL) or representational state transfer (ReST) are also possible.
  • Active storage over the long term of historic and near-current data. Active storage as opposed to static storage, such as a tape archive. This storage should be optimized for reporting and analysis through both its logical and physical data models, and through the database architecture and technologies implemented. Today we're seeing an amazing surge of data storage and management innovation, with column-store relational database management systems (RDBMS), map-reduce (M-R), key-value stores (KVS) and more, especially hybrids of one or several of old and new technologies. The innovation is coming so thick and fast, that the terminology is even more confused than in the rest of the BI world. NoSQL has become a popular term for all non-RDBMS, and even some RDBMS like column-store. But even here, what once meant No Structured Query Language now is often defined as Not only Structured Query Language, as if SQL was the only way to create an RDBMS (can someone say Progress and its proprietary 4GL).
  • Tools for reporting including gathering the data, performing calculations, graphing, or perhaps more accurately, charting, formating and disseminating.
  • Online Analytical Processing (OLAP) also known as "slice and dice", generally allowing forms of multi-dimensional or pivot analysis. Simply put, there are three underlying concepts for OLAP: the cube (a.k.a. hypercube, multi-dimensional database [MDDB] or OLAP engine), the measures (facts) & dimensions, and aggregation. OLAP provides much more flexibility than reporting, though the two often work hand-in-hand, especially for ad-hoc reporting and analysis.
  • Data Mining, including machine learning and the ability to discover correlations among disparate data sets.

For our purposes, an important question is whether or not there are open source, or at least open source based, solutions for all of these capabilities. The answer is yes. As a matter of fact, there are three complete open source BI Suites [there were four, but the first, written in PERL, the Bee Project from the Czech Republic, is no longer being updated]. Here's a brief overview of SpagoBI, JasperSoft, and Pentaho.

Capability SpagoBI JasperSoft Pentaho
ETL Talend Talend
JasperETL
KETTLE
PDI
Included
DBMS
HSQLDB
MySQL
Reporting BIRT
JasperReport
JasperReports
iReports
jFreeReports
Analyzer jPivot
PaloPivot
JasperServer
JasperAnalysis
jPivot
PAT
OLAP Mondrian Mondrian Mondrian
Data Mining Weka None Weka

We'll be using Pentaho, but you can use any of the these, or any combination of the OSS projects that are used by these BI Suites, or pick and choose from the more than 60 projects in our OSS Linkblog, as shown in the sidebar to this blog. All of the OSS BI Suites have many more features than shown in the simple table above. For example, SpagoBI has good tools for geographic & location services. Also, JasperSoft Professional and Enterprise Editions have many features than their Community Edition, such as Ad Hoc Reporting and Dashboards. Pentaho has a different Analyzer in their Enterprise Edition than either jPivot or PAT, Pentaho Analyzer, based upon the SaaS ClearView from the now-defunct LucidEra, as well as ease-of-use tools such as an OLAP schæma designer, and enterprise class security and administration tools.

Data warehousing using general purpose RDBMS systems such as Oracle, EnterpriseDB, PostrgeSQL or MySQL, are gradually giving way to analytic database management system (ADBMS), or, as we mentioned above, the catch-all NoSQL data storage systems, or even hybrid systems. For example, Oracle recently introduced hybrid column-row store features, and Aster Data has a column-store Massive Parallel Processing (MPP) DBMS|map-reduce hybrid [updated 20100616 per comment from Seth Grimes]. Pentaho supports Hadoop, as well as traditional general purpose RDBMS and column-store ADMBS. In the open source world, there are two columnar storage engines for MySQL, Infobright and Calpont InfiniDB, as well as one column-store ADBMS purpose built for BI, LucidDB. We'll be using LucidDB, and just for fun, may throw some data into Hadoop.

In addition, a modern DSS needs two more primary capabilities. Predictives, sometimes called predictive intelligence or predictive analytics (PA), which is the ability to go beyond inference and trend analysis, assigning a probability, with associated confidence, or likelihood of an event occurring in the future, and full Statistical Analysis, which includes determining the probability density or distribution function that best describes the data. Of course, there are OSS projects for these as well, such as The R Project, the Apache Common Math libraries, and other GNU projects that can be found in our Linkblog.

For statistical analysis and predictives, we'll be using the open source R statistical language and the open standard predictive model markup language (PMML), both of which are also supported by Pentaho.

We have all of these OSS projects installed on a Red Hat Enterprise Linux machine. The trick will be to get them all working together. The magic will be in modeling and analyzing the data to support good decisions. There are several areas of decision making that we're considering as examples. One is fairly prosaic, one is very interesting and far-reaching, and the others are somewhat in between.

  1. A fairly simple example would be to take our blog statistics, a real-time stream using SQLstream's Twitter API, and run experiments to determine whether or not, and possibly how, Twitter affects traffic to and interaction with our blogs. Possibly, we could get to the point where we can predict how our use of Twitter will affect our blog.
  2. A much more far-reaching idea was presented by Ken Winnick to me, via Twitter, and has created an on-going Twitter conversation and hashtag, #BPgulfDB. Let's take crowd sourced, government, and other publicly available data about the recent oilspill in the Gulf of Mexico, and analyze it.
  3. Another idea is to take historical home utility usage plus current smart meter usage data, and create a real-time dashboard, and even predictives, for reducing and managing energy usage.
  4. We also have the opportunity of using public data to enhance reporting and analytics for small, rural and research hospitals.

OSS DSS Formalization

The next step in our open source solutions (OSS) for decision support systems (DSS) study guide (SG), according to the syllabus, is to make our first decision: a formal definition of "Decision Support System". Next, and soon, will be a post listing the technologies that will contribute to our studies.

The first stop in looking for a definition of anything today, is Wikipedia. And indeed, Wikipedia does have a nice article on DSS. One of the things that I find most informative about Wikipedia articles, is the "Talk" page for an article. The DSS discussion is rather mild though, no ongoing debate as can be found on some other talk pages, such as the discussion about Business Intelligence. The talk pages also change more often, and provide insight into the thoughts that go into the main article.

And of course, the second stop is a Google search for Decision Support System; a search on DSS is not nearly as fruitful for our purposes. :)

Once upon a time, we might have gone to a library and thumbed through the card catalog to find some books on Decision Support Systems. A more popular approach today would be to search Amazon for Decision Support books. There are several books in my library that you might find interesting for different reasons:

  1. Pentaho Solutions: Business Intelligence and Data Warehousing with Pentaho and MySQL by Roland Bouman & Jos van Dongen provides a very good overview of data warehousing, business intelligence and data mining, all key components to a DSS, and does so within the context of the open source Pentaho suite
  2. Smart Enough Systems: How to Deliver Competitive Advantage by Automating Hidden Decisions by James Taylor & Neil Raden introduces business concepts for truly managing information and using decision support systems, as well as being a primer on data warehousing and business intelligence, but goes beyond this by automating the data flow and decision making processes
  3. Business Intelligence Roadmap: The Complete Project Lifecycle for Decision-Support Applications by Larissa T. Moss & Shaku Atre takes a business, program and project management approach to implementing DSS within a company, introducing fundamental concepts in a clear, though simplistic level
  4. Competing on Analytics: The New Science of Winning by Thomas H. Davenport & Jeanne G. Harris in many ways goes into the next generation of decision support by showing how data, statistical and quantitative analysis within a context specific processes, gives businesses a strong lead over their competition, albeit, it does so at a very simplistic, formulaic level

These books range from being technology focused to being general business books, but they all provide insight into how various components of DSS fit into a business, and different approaches to implementing them. None of them actually provide a complete DSS, and only the first focuses on OSS. If you followed the Amazon search link given previously, you might also have noticed that there are books that show Excel as a DSS, and there is a preponderance of books that focus on the biomedical/pharmaceutical/healthcare industry. Another focus area is in using geographic information systems (actually one of the first uses for multi-dimensional databases) for decision support. There are several books in this search that look good, but haven't made it into my library as yet. I would love to hear your recommendations (perhaps in the comments).

From all of this, and our experiences in implementing various DW, BI and DSS programs, I'm going to give a definition of DSS. From a previous post in this DSS SG, we have the following:

A DSS is a set of processes and technology that help an individual to make a better decision than they could without the DSS.
-- Questions and Commonality

As we stated, this is vague and generic. Now that we've done some reading, let's see if we can do better.

A DSS assists an individual in reaching the best possible conclusion, resolution or course of action in stand-alone, iterative or interdependent situations, by using historical and current structured and unstructured data, collaboration with colleagues, and personal knowledge to predict the outcome or infer the consequences.

I like that definition, but your comments will help to refine it.

Note that we make no mention of specific processes, nor any technology whatsoever. It reflects my bias that decisions are made by individuals not groups (electoral systems not withstanding). To be true to our "TeleInterActive Lifestyle" &#59;) I should point out that the DSS must be available when and where the individual needs to make the decision.

Any comments?

Why Open Source for a Friend

This post is in response to "Volunteer for the Greater Good" written by S. Kleiman. I remember that village in Pennsylvania, and the attitudes of my friend at that time. I'm not surprised that you're attracted to open source; I am surprised that you're having trouble with embracing its ideals. We've have had an email exchange on this subject, and, as you know, I'm fairly attracted to open source solutions my self. &#59;) I hadn't seen your blog prior to answering your email, so let me go into a bit more detail here.

"The model contributor is a real geek – a guy in his 20-30’s, single, lives in his parent’s basement, no mortgage, no responsibility other than to pick up his dirty socks (some even have mothers who will do that)." -- "Volunteer for the Greater Good" by S. Kleiman

Wow. What a stereotype, and one that couldn't be further from the truth. Admittedly, during economic downturns, when software developers are forced to take whatever job they can find to put food on the table, many contribute to open source projects, ones that don't have commercial support and ones that do. This helps that open source project and its community. But, it also helps the developers to keep their skills sharp and maintain credibility. Most open source developers get paid. Some are students. Some are entrepreneurs. But most get paid, it's their job. And even if it's not their job, projects have learned to give back to communities.

While there are hundreds of thousands of open source projects on Sourceforge.net and other forges, many have never gone beyond the proposal stage, and have nothing to download. The number of active open source projects does number in the tens of thousands, and that is still pretty amazing. The idea that the great unwashed contribute to these projects whilst Mom does laundry... Well, that just doesn't wash. :p The vast majority of open source communities are started by 1 - 5 developers, who have a common goal that can be obtained through that specific open source project. They have strict governance in place to assure that the source code in the main project tree can be submitted only by those that have founded the project, or those that have gained a place of respect and trust in the community (a meritocracy) through the value of the code that they have contributed for plugins, through forums, and the like.

Most active open source projects fall into two categories, and many have slipped back and forth between these two.

  1. A labour of love, creating something that no one else has created for the sheer joy of it, or to solve a specific pain point for the lead developer
  2. A commercial endeavor, backed by an organization or organizations to solve their own enterprise needs or those of a specific market

While there are thousands of examples of both types, let me give just a few examples of some developers that I know personally, or companies with which I'm familiar.

Mondrian was founded by Julian Hyde, primarily as a labour of love. I know Julian, and he's an incredibly bright fellow. [And public congratulations to you, Julian and to your wife, on the recent birth of Sebastian]. In addition to be the father of Sebastian and Mondrian, Julian is also the Chief Architect of SQLstream, and a contributor to the Eigenbase project. Not exactly sitting around in the basement, coding away and waiting for Mom to clean up after him. :>> You can read Julian's blog on Open Source OLAP and Stuff, and follow Julian's Twitter stream too. By the way, while Mondrian can still be found on Sourceforge.net under its original license, it is also sponsored by Pentaho, and can be found as Pentaho Analysis, and as the analytical heart of the Pentaho BI Suite, JasperSoft BI Suite and SpagoBI.

Two other fellows had somewhat similar problems to solve and felt that the commercial solutions designed to move data around were simply too bloated, too complex, and prone to failure to boot. I don't believe that these two knew each other, and their problems were different enough to take different forms in the open source solutions that they created. I'm talking about Matt Casters, founder of the KETTLE ETL tool for data warehousing, and Ross Mason, founder of the Mule ESB. Both of them had an itch to scratch, and felt that the best way to scratch it was to create their own software, and leverage the power of the open source communities to refine their back scratchers. KETTLE, too, can now be found in Pentaho, as Pentaho Data Integration. Ross co-founded both Ricston and MuleSource to monetize his brain child, and has done an excellent job with the annual MuleCons. Matt still lives in Belgium, and has been known to share the fine beers produced by a local monastery [Thanks Matt]. You should follow Matt's blog too. Ross lives on the Island of Malta, and Ross blogs about Mule and the Maltese lifestyle.

Let's look at two other projects: Talend and WSO2. Both of these are newer entrants into the ETL and SOA space respectively, and both were started as commercial efforts by companies of the same name. I haven't had the opportunity to sit down with the Talend folk. I have spoken with the founders of WSO2, and they have an incredible passion that simply couldn't be fulfilled with their prior employer. So they founded their company, and their open source product, and haven't looked back. You can follow Sanjiva's Blog to learn more about WSO2 and their approach to open source.

And just one more, and somewhat different example: projects started by multiple educational institutions to meet their unique needs: Kuali for ERP and Sakai for learning management. For another take on commercialization, The rSmart Group contributes to these projects, but is commercializing them as appliances sold to educational institutions. You can read more about this rather different approach to monetizing open source at Chris Coppola's blog.

There are many, many more such examples. Just in the area of data management & analysis, we cover over 60 related open source projects [take a look at the blogroll in the sidebar to the right.

..."they organize themselves into groups of developers and maintainers on an adhoc basis, and on a world-wide basis. And the end products are robust, well developed, and well tested." -- "Volunteer for the Greater Good" by S. Kleiman

I think we've covered my rebuttal to your posting between the first quote and this one. I very much agree with this statement. I'm surprised by your surprise. The organizational dynamics that result in the excellent code that comprise open source projects is the subject of much thought, admiration and research. Here's a few places that you can go for more information.

And just for completeness sake, here's our email exchange:

From S. Kleiman: "OS is the current bug in my head. I'm trying to understand why my intellectual property should be "open" to the world (according to Richard Stallman.

Yes, I've read the copious amounts of literature on open software and the economics thereof - but I still don't get it. If I apply for a patent on a gadget, and then license companies to make that gadget - isn't that intellectual property? To copy my design, while it doesn't destroy my design, does limit any profit I might gain.

Anyway - how are you? Are you one of the original hackers?
I realized that all this time I though I had a great practical engineering degree. Instead I realize they made us into hackers - in the best sense of the word.

What is your experience with OS? What are you talking about (besides the title)?
How is the "snow" in CA? "

And my response:

Discussions around open source often get very passionate, so we should be having this conversation on a warm beach cooled by ocean breezes, fueled with lots of espresso ristretto followed by rounds of grappa to lower inhibitions and destroy preconceptions ;-)

But email is all we have.

Most open source projects are software, though there are a few examples of hardware projects such as Bug Labs, TrollTech (bought by Nokia, I think), OpenMojo and one for UAVs.

I should start by pointing out that I'm not presenting at the Open Source Business Conference, but am moderating a panel.

http://www.infoworld.com/event/osbc/09/osbc_agenda.html

Session Title: Moving Open Source Up the Stack

Session Abstract: Open Source Solutions for IT infrastructure have shown great success in organizations of all types
and sizes. OSS for business applications have seen greater difficulties in penetrating the glass ceiling
of the enterprise stack. We have put together a panel representing the EU and the US, system
integrators, vendors and buyers, and corporate focus vs. education focus. We''ll explore how the OSS
application strategy has changed over the past four years. We will also look at success and failures,
the trade-offs and the opportunities in solving business/end-user needs with OSS enterprise
applications.

Learning Objective 1: Most buyers know the 80% capability for 20% cost mantra of most OSS vendors, but we''ll focus on
what that lower cost actually buys.

Learning Objective 2: Where does OSS fit in the higher levels of the application stack? Learn how flexibility & mashups
can improve the end user experience.

Learning Objective 3: Learn how to come out ahead on the trade-offs of up-front cost vs. operational cost, experience and
learning curves, maintenance and replacement, stagnation and growth.

Here are the confirmed panelists:

(1) Tim Golden, Vice President - Unix Engineering, Security & Provisioning, Bank of America
(2) Gabriele Ruffatti, Architectures & Consulting Director, Research & Innovation Division, Engineering Group, Engineering Ingegneria Informatica S.p.A.
(3) Aaron Fulkerson, CEO/Founder, mindtouch
(4) Lance Walter, Vice President - Marketing, Pentaho
(5) Christopher D. Coppola, President, The rSmart Group
(Moderator) Joseph A. di Paolantonio, Principal Consultant/Blogger/Analyst, InterActive Systems & Consulting, Inc.

So, back to the "Why open source" discussion.

You might want to listen to a couple of our podcasts:

http://press.teleinteractive.net/tialife/2005/06/30/what_is_open_source

http://press.teleinteractive.net/tialife/2005/07/01/why_open_source

or not :-D

Historically, there were analog computers programmer by moving around jumper cables and circuits. Then there were general purpose computers programmed in machine language. Companies like IBM got the idea of adding operating systems, compilers and even full applications to their new mainframes to make them more useful and "user friendly" with languages like COBOL for the average business person and FORTRAN fir those crazy engineers. Later Sun, Apple, HP and others designed RISC based CPU's with tightly integrated operating systems for great performance. Throughout all this, academicians and data processing folk would send each other paper or magnetic tapes and enhance the general body of knowledge concerning running and programming computers. There eventually grew close to 100 flavours of Unix, either the freely available BSD version or the more tightly licensed AT&T version.

Then a little company called Microsoft changed the game, showing that hardware was a commodity and the money was in patenting, copywriting and using restrictive licenses to make the money in computers come from software sales.

Fast forward ~15 years and the principals in Netscape decided to take a page from the Free Software Foundation & their GNU (Gnu is not Unix) General Public License and the more permissive Berkeley License for BSD and as a final recourse in their lost battle to the Microsoft monopoly, coined the term "open source" and released the geiko web rendering engine under the Mozilla Public License. And the philosophical wars were on.

When I was the General Manager of CapTech IT Services, I had a couple of SunOS Sys Admins who spent their spare time writing code to improve FreeBSD & NetBSD. I let them use their beach time to further contribute to these projects. Then a young'un came along who wanted to do the same for this upstart variant of minix called Linux. :-D. All of this piqued my interest in F/LOSS.

Today, I feel that F/LOSS is a development method and not a distribution method nor a business model. If you look at IBM, HP, Oracle and others, you'll find that >50% of their money comes from services. Just as M$ commodified hardware and caused the Intel CISC architecture to win over proprietary RISC chips, software has become a commodity. Services is how one makes money in the computer market. With an open source development methodology, a company can create and leverage a community, not just for core development but for plugins and extensions, but more importantly that community can be leveraged ad thousands of QA testers at all levels: modules, regression & UAT, for thousands of use cases, and for forum level customer support (People, people helping people, are the happiest people on the world ;-)

Can the functions in your application be replicated by someone else without duplicating a single line of your code? Are the margins on your software sales being forced below 10%? Does most of your profit come from support, system integration, customizations or SaaS? Then why not leverage your community?

So, this is a really short answer to a really complex issue.

To answer some of your other questions...

I'm not an hacker nor a programmer of any type. I have started to
play around with the open source R statistical language to recreate my Objective Bayes assessment technique and grow beyond the (Fortran on OS/360 of VAX/VMS) applications that I caused to be created from it.

I haven't gotten to the snow in a couple of years, but we're in a drought cycle. Though it is storming as I write this.

I hope this helps you with your open source struggle, my friend. And thank you for putting up with me being a wordy bastard for the past /cough /harumph years. :D Oh, and note the Creative Commons license for this post. This must really cause you great consternation as a writer. Oh, and I'm not going to touch your post on Stallman. B)

Making Sense of Open Source

The highlight of JavaOne for me has become supper with Gianugo Rabellino, the founder and CEO of SourceSense. For now, each year…

"'The time has come,' the Walrus said, 'To talk of many things: Of shoes—and ships—and sealing-wax— Of cabbages—and kings— And why the sea is boiling hot— And whether pigs have wings.'"
-- from the poem "The Walrus and the Carpenter" within Through the Looking Glass by Lewis Carrollend quotation

… For our conversations are wide ranging and thoroughly engaging as we indulge our enjoyment of fine food and open source.

This year, Gianugo has been in the USA several times, for the Open Source Think Tank and the Open Source Business Conference, and on business. An employee of SourceSense even had a gig in the town in which I was raised. Unfortunately discovering it to be the arm pit of the United States. Ah well, it's why I live in the SF Bay Area now. &#59;)

We talked of tempering chocolate, unusual eating habits of various cultures, the worsening economy, European vs USA political views, and Gianugo even had me read, while sober, the Pronunciation Poem. [I flubbed a few words, and disagree with some of the rhymes given that my dialect is as Philly as a Chesse Steak.]

But we also talked about Open Source; all the open source related conferences and blogging and work and newsworthy activities of the past few months. Here are some of my thoughts.

Open Source is a Philosophy

This one has been bubbling around in my head for years. Open Source, in and of itself is NOT

  • a licensing template
  • a business model
  • a development methodology

It is a philosophy that can provide the framework for those three things with which it is most often identified. The variations among what the open source philosophy means to each of its followers can best be seen from the proliferation of licenses, business models and methodologies all claiming to be open source.

To me, the open source philosophy is very simple, and it can be applied to solutions for some very complex problems and concerns: the source is available to anyone who obtains the end-product, whether one has obtained the end-product through a no-cost download or through a purchase agreement of whatever type. The "source" may be the "source code" for software, but, to me, it should be whatever specifications and design documentation are required to recreate the end-product in either the original or a modified form.

There are a variety of reasons that a developer, an engineer, an inventor, a creator, might what to subscribe to an open source philosophy. And each individual or business must decide if those reasons make creative and economic sense for them. Once one has decided to subscribe to open source, or any philosophy, then all your other decisions will only lead to success if they are logically consistent within that philosophy. The licensing language, business model, economic forecasting, internal processes and external relationships should form a coherent whole within the underlying philosophical framework.

Business Models Will be Different for Different Markets

Organizations often look for the silver bullet or the golden mantra or the platinum ring that will solve their problems, lead to dominance in the marketplace, or allow them to rule them all. &#59;) So we hear a lot of talk about the "best" open source business model. This search ignores that fact that there are open source solutions for many markets. Without going into specific verticals, let's just consider four general, horizontal markets that are addressed by open source software.

  1. Information Technology Infrastructure: including operating systems such as the various flavours of linux and BSD unix, application servers, and other middleware such as the Mule ESB, WSO2 SOA solutions & KETTLE for ETL, web servers, email MTA, the Funambol Mobile Server platform, and many more.
  2. End User Applications on the Desktop/Laptop/Mobile-Device: with OpenOffice.org and its MacOSX offshoot NeoOffice, Projivity OpenProj, and Mozilla Firefox and Thunderbird likely being the most well known, but also including many thousands of open source projects for artistic creation and enjoyment, as well as office and personal productivity, and gaming, and all things that one personally does on a computer
  3. Software Development Tools: from the Eclipse IDE to hundreds and thousands of specific language libraries and everything else a developer might need; this is even more "by geeks for geeks" than the infrastructure area
  4. Enterprise Applications: these are end-user facing applications, often fulfilling mission critical needs, such as ERP, CRM and BI, and is the least mature market for open source providers and the hardest area to gain acceptance

No one business model, no matter how generically expressed, could satisfy these four disparate markets. Monetizing these areas will come from combining innovative and traditional packaging of support, customizations, system integration, training, licensing, and subscriptions.

Automated maintenance, repair and update networks might work very well for monetizing IT infrastructure, but might be insufficient for the other markets. Ad based monetization, directly or through partnerships with ad networks like Google's, might work for some end-user applications. Putting out the begging bowl, asking for PayPal contributions might also work. But for many open source projects, there simply isn't any path to monetization.

One area that I think has been insufficiently explored, and might well be the only path to success for the Enterprise Application open source vendors is Software as a Service. The SaaS approach, whether through partner channels or directly, is the most sensible means of monetizing a wide variety of open source applications. Embrace the open source philosophy, leverage the strengths of flexibility and community in the licensing, business models and processes, and monetize through SaaS delivery into vertical and niche markets.

And for the rest, just acknowledging that your company is a software company embracing an open source philosophy, and building appropriate support and licensing structures will be the best path.

Communities are Strongest when Open

Developing a community around a product is not unique to the open source world. Microsoft, Oracle, SAP, SAS, SPSS, IBM, etc, etc, etc have all developed and supported great communities made up of ISVs, VARs and Users. The difference however, is that open source projects are very dependent on their community. Conversely, communities are strongest when there are no artificial limits to communication.

I think that most companies are embracing at least this one ramification of the open source philosophy. Though it seems to me that when the source isn't open, the community can only converse and support each other in surface issues.

One area though, where open source vendors still seem to be uncertain, is the definition of community, and what groups make up their community, and if they must develop different communities for different categories of members: developers, users, non-software contributors, paying customers, or partners. I think this is self-defeating. There is only one community for each project: those who want to be involved and will benefit from that involvement, ultimately to the betterment of the project, whether directly or indirectly, whether monetary or not, whether contributing code or documentation or use cases or testing or playing around with the project or complaining about it or just soaking up the ambience. Can any open source project community manager give a valid reason why any of these should be excluded from the community or ignored?

Open Source Sales are Very Foreign to Most IT Buyers

Whether we're talking about senior IT management, the CIO, or the purchasing department, the idea of freely downloading the fully operational, unlimited version of a product, and using it for research, prototyping or production, without handholding, cosseting or sales incentives is complete anathema to those making purchasing decisions.

Did you ever see a "vendor bake-off" among products from Oracle, IBM and SAP without vendor involvement, without vendor sales engineers fine-tuning and providing weeks of free labour? Is there any open source company that provides a corporate jet and box seats at the super bowl? And is there anyone who has ever been part of selling (internally or as a vendor) a large, big-budget enterprise project to a CIO or purchasing committee, who doesn't believe that such freebies and perks are essential elements to the approval process?

Many, too many, open source vendor CEOs believe that bloatware (more features than most users will use), vendor lock-in and lack of interoperability, and high prices of initial licenses are the problems CIOs worry and that open source software solves. They think success will come from "80% of the capabilities for 20% of the cost" of their closed source competitors. I don't believe any of it. CIOs worry about being beat up by the business side and greasing the squeaky wheel. The two highest costs in a data center are people and energy, not licenses or maintenance agreements. Implementation costs: hardware, software licenses, personnel and training, are spread out over three to five years with appropriate tax implications. For really large projects such as a full ERP or data warehouse, there is at least a ten year life expectancy. Why do you think that there are so many mainframes and COBOL applications still around? The implementation costs were written-off long ago, and the on-going support costs are minimal because they still JUST WORK. Those with an eye to economics are more concerned about the total cost of ownership over the complete 10-year lifecycle of a capital project, than with the 10% or less of those costs that go towards initial software licenses.

So, if the only way for open source vendors to become big players, or as one CIO put it, to "move out of the junior varsity", is to look like the big players, to provide pre-sales engineering and collateral beyond a T-shirt, then what will this do the business model and the belief in "20% of the cost"? It will destroy it. This is why there are more openings for sales & marketing than for engineers at open source companies today.

Open Source Vendors will grow and evolve and come to operate more and more like the big players they're destined to become. As long as they continue to adhere to their form of the open source philosophy, are internally consistent and true to the logic of their open source framework, they'll continue to benefit from the true value of open source: flexibility and community knowledge to a greater degree than can be achieved by any closed-source vendor or any isolated, enterprise data center with home-grown solutions. Speaking of which…

Open Source is the Middling Ground between Build vs Buy

This is another point I've been hammering for years. Traditionally, IT shops were either build or buy. The build shops were very development oriented and created custom solutions to implement business processes and support business users. The buy shops bought COTS software, and either convinced the business that the software they bought implemented industry best practices, or spent millions of dollars in customization.

Build shops suffer from isolation and the total burden of maintaining and updating the software they built. When the only developer left who remembers and understands the code, retires, faith often becomes the best practice for ongoing support. :>>

Buy shops suffer from ongoing dependancy on the vendor(s) and ongoing compromises for the users.

Build shops should be (but often aren't) more flexible to respond to changing business needs, and can be more valuable to the business.

Buy shops can be more reliable and cost-effective - really. But not always.

One result of Y2K [remember that?] was that most IT departments became much more of a blend of build and buy, with buy decisions winning out. Coupling this with the facilitation of distributed workgroups made possible by the Internet expanding world wide, and offsourcing [outsourcing operations to offshore companies] decimated many corporate data centers by solving the personnel and energy cost problems. Guess what? Implementation and licensing and maintenance costs remain the same. Some CIOs use offsourcing as the reason that they don't use open source; the outsourced vendor isn't familiar with it, and has no incentive to use it.

As companies bring IT back inside, and as its importance to the business is once again realized, open source offers a third path to the traditional build or buy. However, offsourcing will still be a substantial part of the solution, and should lead OSVs to recognize the importance of offsourcing vendors and embrace them as partners and very important channels.

The IT shop that responds to the business using open source can be both flexible and well-supported. The open source vendors that make such IT shops succeed through flexibility and reliability will be the most successful ones. This can be achieved through the OSVs growing their support, professional services and training organizations, or by partnering with all sizes of PS, VAR and outsourcing firms, or both.

Conclusions

The real conclusion is that bringing the open source philosophy to fruition in business is still an evolving process. The advantages gained by the openness of the source and the strength of the community is being recognized by both the IT shops and the OSVs. While there are cost savings to be had, OSVs need to stop relying on initial licensing cost reductions as their main selling point, and begin to market the advantages to the IT shop of using open source: responsiveness to changing business needs and increasing reliability over time, all while providing the best TCO and ROI.

There's a lot more to be said on all of these topics and opinions, and maybe I'll even get the time do so. :p

Startup Camp Difference

Startup Camp is a mix of traditional conference panels and an open space "unconference".

The juxtaposition of the two is a bit jarring in some ways. 'Tis somewhat like going from a warm fuzzy blanket [panels] to an exhilarating plunge into the chilly North Pacific [unconference sessions]. Actually, panels have been losing their appeal over the past few years. A good panel is a lively debate. The panelists engage each other. A good panel is both entertaining and informative. Too many panels today bring to mind an event from my college days. This is an unfair analogy but I'll tell the story anyway. &#59;)

The only diner within walking distance of the college was the S&W [or, as it was traditionally known, the "Slut & Whore" - hey, don't yell at me; it describes the area well. Think of the Tenderloin in SF on a bad night.] Anyway, after an all-night study session, or maybe a drunken game of Diplomacy, we made a visit to the S&W in the wee hours for some coffee and rice pudding. Among the other patrons were two old transients [a.k.a. hobos, stumblebums]. They were at opposite ends of the diner, but seemed to be having a loud argument on who was the greatest baseball player of all time. After a while, we realized that they weren't arguing. They were merely proclaiming in loud voices on the same subject. OK, the panels aren't that bad, but getting close.

Another problem with panels today is that most panelists blog. If you are at all interested in the subject, it's likely that you read their blogs and already know their opinions.

Ah well, onto the good part.

The unconference sessions are intimate and, as mentioned before, exhilarating.

The only problem is that there are so many great topics being discussed at the same time, that it's like being in a candy store: you don't what to grab onto, there are just too many choices.

But you can read all about what happened on the Startup Camp wiki, if the attendees upload their notes. :D Just follow the links from the Startup Camp Unconference schedule at the bottom of the page.

Tomorow will be running back and forth between Startup Camp and CommunityOne. Can't wait.

Startup Camp 2008 Sunday Morning

Today, I'm at Startup Camp, in conjunction with CommunityOne, which starts tomorrow, in conjunction with JavaOne. The schedule is online, but this is a camp, an unconference, and the rules are different.

The introduction is over, and the keynote with Jonathan Schwartz and Om Malik is going on now.

A question from the audience, essentially that there is no simple, online solution for a Java developer to just go online and develop, led to an interesting side discussion where I'm seated about what it means to be a developer. The old folk at the table, have a much broader definition of a developer, and the skills they should have, than the young'uns, who are more focused on just one language, and just code monkey banging away.

The first panel is starting up, moderated by S. Neil Vineberg, President, Vineberg Communications. The topic is branding, and how the brand comes out of the founders and the culture that they create.

One interesting point for me, is that while the panel is discussing the importance of branding, and that the brand can flow from the founders' personality and the culture, I have found that if the founders are too focused on creating a brand, and selecting their brand category, they'll fail to infuse their brand with their personality, that is, the brand can seem sterile and contrived. Branding is very important, but, to me, it must come naturally from the company if it is to truly reflect the company. Of course, sometimes you have a founder that you need to keep locked in a closet, feed caffeinated drinks and pizza, and never, ever let them talk to the customers. :>>

The use of social media is a given today, especially for startups. The use of Ning [why doesn't PeopleAggregator ever get a mention?] and the growth of social network platforms for startups [others] to grow their own social networks is a great indicator of this.

For all the talk of social media, the message of this panel seems to be that traditional methods, through PR, is still the best way to reach out and get your message across; especially outside of the technology centers like the SF Bay Area. However, it seems to me that when the panel starts talking about what's really effective, they use terms like "community" and "authentic conversations". It also seems that creating markets and driving markets is the better than pushing a product, in terms of success, and in terms of getting noticed.

The morning is done. I'm looking forward to networking and learning throughout today and tomorrow, to CommunityOne and seeing Michael Coté at Redmonk's Unconference, and a great conversation with Gianugo Rabellino at JavaOne and for supper on Thursday.

OSBC2008 Presentations Downloads

InfoWorld has made downloads available for selected presentations from this year's Open Source Business Conference. The links will take you to PDF files. But, Matt, where are your video mashups?

MuleCon2008 Closing Campground

The community has been brought together for the final campground session. Rather than sing kumbaya, we've been teased with t-shirts, toys and books from O'Reilly and being treated to a demo of a hot deployment by Travis Carlson. 'Tis all drag-and-drop goodness, with dragging jar files (bundles) as connectors and apps from a MacOSX Finder into Mule and watching the results in Terminal.app. Ohhh! Ahhh! Safaris shows Mule saying "Hello Travis, welcome to MuleCon" B) OK, I'll admit that this is pretty neat and should satisfy the most addicted user of Tibco wizards. This won't appear in Mule Enterprise Edition until after the 2008Q3 release of Mule EE 2.0.

One little aside… I was talking to an ex-TIBCO employee who said that as old as TIBCO is (grew out of a Teknecron business that was founded in 1985 and became an independent business in 1997), the user even only draws about 400 people. This is only the second MuleCon [there was a sort-of pre-MuleCon in 2006 at a bar in London, but let's not count it), which last year drew around 100 folk and this year brought in around 240. That's phenomenal growth and shows the excitement that good software can bring to its users.

Questions for the Campground from Day 1 and some new ones.

  1. AMQP is an emerging messaging protocol and is platform independent. To answer the question about dot-Net integration with Mule, this is part of the answer.
  2. BPEL & BPM? Workflow tools, such as JBPM, that don't depend on simple web services, work better with Mule. Travis has done some work with this including the JBPM plugin that is in the current distribution of Mule, and there is more on the Forge. Mule also works well with Oracle and Iona's solutions. Mule Services and Mule Events are complementary.
  3. Mule is moving to Spring2, and has MuleSource thought about the exciting things that can be done with AOP in Spring2? Not so much yet, but it is among the next thing to look at. Mule Galaxy might be one of the first areas where MuleSource would leverage AOP to create runtime governance.
  4. Can Mule Galaxy can be used to manage other artifacts? The short answer is yes.
  5. How are folk clustering Mule? The typical way is to leverage an existing clustered app server. They are seeing more where the ESB has a state itself. They're looking at things like TerraCotta for such use cases.
  6. Discussions of rules engines and routing, but much work to be done, hopefully through the MuleForge.

Dave gave some closing remarks and that's all 'til next year.

MuleCon2008 Users Day 2

This first user track session is a panel, once again moderated by Michael Coté of Redmonk with John Davies, Technical Director and Head of Research at Iona, Eugene Ciurana of Leapfrog, John Rowell, CTO of OpSource, and John Gardner, Principal Consultant at MomentumSI. My take away from this is that Mule, perhaps open source in general, allows a company to better balance their risk as they test out the potential rewards while implementing tactically against strategic goals. A great metric from Eugene is that the most wait time he has ever had with Mule for a support issue is 90 minutes from opening a ticket to getting some sort of answer, and this is what keeps selling him on Mule vs proprietary solutions. Cost is a factor in ROI, and Mule wins out on not just initial costs but also costs less in need for training, on-going support, consulting, and maintenance; but beyond this, the flexibility of Mule and ability to integrate it with the infrastructure, and the openness not just of the code but in the company and personnel also adds to the return, as does the "cool" factor.

Having now seen Coté moderate two panels now, I have to say that he's one of the better moderators I've seen.

The second session is "A Quantum Leap in Ease of Use: Introducing Mule IDE 2.0". First let me say that having a nice, GUI IDE will help with the ROI perception, in that it is often the perception that development can be quicker and more efficient if done in a nice, pretty GUI. I happen to disagree, but I'm old and like CLI and typing more than clicking. Let's see what these guy's say. Yep, yep, yep… they're showing a pretty drag-and-drop GUI configuration editor. I took the one-day training at last year's MuleCon, and found that configuration through editing XML files to be very easy, and to help with understanding what Mule was doing. There was a contingent of attendees who complained that they didn't have their Tibco Wizard. This should help placate that contingent. Ah, the Mule IDE 2 is pre-alpha, though you can get it on the Forge. The release is expected to be in 2008Q3. The demonstration of the Mule IDE 2.0 plugin for Eclipse looks good; in addition to the GUI, one can still see the underlying XML.

John Davies, Technical Director and Head of Research of Iona spoke on Financial Messaging in Mule. The concept of losing a message in an investment is literally unthinkable. One interesting tidbit is that MS Excel is the only MS products one finds in an investment bank. John went over the front/middle/back office requirements, and the place where Mule is found is in the middle where XML is heavily used, usually over MQ & JMS. Using Artix, Mule, and GigaSpaces on an Azul box, provides essentially linear scaling and throughput is so high, performance is so great, that the need for transactions and supporting transactional relational databases is eliminated, thus increased performance allows increasing performance even more.

Mule or 5th Grader Who's Smarter

The final session of MuleCon2008's second morning is a panel based upon "Are you Smarter that a Fifth Grader", moderated my Michael Coté of Redmonk. The panel consisted of Dave Rosenberg, Larry Augustin, Jason Maynard and Matt Asay.

The first question was what gets these folk excited in the shower in the morning: SaaS, Web2.0 enterprise/consumer Mashups, or other newly hyped trend. SaaS and mashups seem to be joined in the answers as a way for business users to get around IT by buying what they need on a credit card and expensing back this "rogue IT" service. The question evolved into "Is open source a given?". The answer was "yes" and the discussion segued back into how all these things, SaaS, mashups, open source, allow business users to get what they need quickly. Coté then asked if this meant that there was a "work life retooling" going on that was allowing traditional IT departments to work with Rogue IT implementations. While the consensus seems to be that IT departments were indeed going through the work life retooling to allow Rogue IT implementations, such as with WiFi hot spots or open source, I have to disagree. Every IT group with which we've worked has many regulatory, compliance or responsibility issues that preclude them from allowing Rogue IT. However, if a technology shows its value, even if that value can not be quantified in simple cost-benefits analyses, that technology will be brought into the organization under the IT umbrella, one way or another.

The floor was opened to questions from the attendees.

I asked if the panel would discuss MDM being useless without SOA and vice-versa. They pretty much declined, but Jason did discuss some of his own experiences. The bottom line is that the concepts are difficult, and are only starting to be embraced by developers and companies.

Another question asked why there seems to be so little uptake of open source ERP. I would like to ask the same about OSBI, but didnt' need to do so, as Matt brought it up. At this point, open source seems to fill niches within a given enterprise, and one can build a niche OSBI solution more easily than a niche ERP solution. Larry also pointed out that many large organizations in the US have ERP implemented, and they're not going to rip it out to go open source; however, if you look at other markets, where ERP implementations didn't happen [as part of the Y2K fixes], you can find uptake of open source ERP.

Another attendee pointed out the only the oldest dinosaur can question the quality of open source, yet one still must go underground to implement ERP in most organizations. The panel discussed the difficulties of getting through procurement at a large company, and how tactical implementations [back to rogue IT] rather than strategic are often more plausible. It seems to be a matter of market education, and that takes time. Matt also talked about the post-procurement process, and how that is made easier by open source in that the risk is much less.

BPM, SOA, real-time-enterprise… This nirvana, or any other TLA, doesn't really exist in any enterprise, and many of these concepts are hard sells, whether implemented with open source solutions or not.

December 2018
Mon Tue Wed Thu Fri Sat Sun
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31            
 << <   > >>

At the beginning, The Open Source Solutions Blog was a companion to the Open Source Solutions for Business Intelligence Research Project, and book. But back in 2005, we couldn't find a publisher. As Apache Hadoop and its family of open source projects proliferated, and in many ways, took over the OSS data management and analytics world, our interests became more focused on streaming data management and analytics for IoT, the architecture for people, processes and technology required to bring value from the IoT through Sensor Analytics Ecosystems, and the maturity model organizations will need to follow to achieve SAEIoT success. OSS is very important in this world too, for DMA, API and community development.

37.652951177164 -122.490877706959

Search

  XML Feeds

mindmaps

Our current thinking on sensor analytics ecosystems (SAE) bringing together critical solution spaces best addressed by Internet of Things (IoT) and advances in Data Management and Analytics (DMA) is here.

Recent Posts

Run your own website!