SQLStreamv2 Real Time BI

Today, SQLStream announced version 2.0 of their Real Time BI solution. SQLStream comes from the fertile creativity of Julian Hyde, who is also the founder of the open source Mondrian OLAP engine. While SQLStream is not open source, it does stem from the open source Eigenbase community, leveraging the user-defined transforms that were originally developed for LucidDB to operate on traditional stored relational data, with SQL:2003-compliant syntax. SQLStream extends this to handle streaming relational data.

In addition to capturing standard, structured data while "on the wire", SQLStream also includes adapters for feeds, such as Atom and RSS, and for Twitter.

Methinks Julian and I need to schedule another lunch soon, so that I can learn more about how this unstructured data, especially from Twitter, can fit into real time analytics provided by SQLStream v2.0.

BTW, you can follow me on Twitter as @JAdP.


We'll be leading a session at the SOA Consortium Meeting being held at the Santa Clara Hyatt Regency on 2008 December 10 & 11. I'm saying "leading a session" because, as opposed to the normal slide deck in MS PowerPoint, OpenOffice.org/NeoOffice Impress or Apple Keynote, we'll be using a mindmap to, as the agenda says:

"An interactive session based upon a mindmap for developing a system architecture using Master Data Management (MDM), Service-Oriented Architecture (SOA) and Software as a Service (SaaS) principles. The goal is not to talk about having an enterprise mashup with salesforce.com, but how to apply these principles to internal enterprise initiatives. We'll discuss the success, lessons learned and future of integrating MDM & SOA, and how this approach allows IT to provision business needs quickly through a SaaS approach to the users. Bring your own experiences and ideas, as we'll be expanding the mindmap in the direction you want. A PDF of the basic mindmap will be emailed to all members of the SOA-C and be included in the meeting handout. Changes to the mindmap made during the session will be posted after the meeting."

As consultants, we like to listen &#59;) and as believers in the power of collaboration, we like to leverage the wisdom of the group rather than pontificate from a podium. Indeed, this opportunity to speak came through interactions on Twitter, the so-called micro-blogging 24x7 TeleInterActive conversation. Thanks to Brenda Michelson, or @bmichelson by Twitter handle, for arranging this opportunity. The result of all this, is that we prefer to have a fully interactive session with the participants. We want the conversation to go in new and interesting directions. The way we do this, is much like the job of a community manager, but at the micro level. We're hoping that all will join in, and it's our job to assure that we maximize the value of the conversation to the group without abusing anyone's comfort level.

The point of this discussion is to explore how the concepts and principles of Master Data Management (MDM), Service-Oriented Architecture (SOA) and Software as a Service (SaaS) can help Information Technology (IT) departments better serve their customers. We'll be exploring how MDM and SOA work in a SaaS environment from our work with several SaaS firms, how SaaS companies leverage these principles to quickly provision and respond to their customers, and how this differs from a traditional IT department responding to business users and bringing them into production.

One of the most important aspects of this area, is the idea of data services, and how Master Data Management works within a Service-Oriented Architecture to give the users what they really need: access to legacy, historical and transient data.

We'll be starting with our MDM, SOA & SaaS mindmap, collapsed to the first level of branches, and following the branches that are of most interest to the participants. We'll be extending and modifying the mindmap as we go along, and posting the revised mindmap on this blog after the session.

Microsoft Acquires Datallegro whither Ingres

I've been "hearing" all day on Twitter that Microsoft would be announcing something big at OSCON2008. Perhaps this is it:

Microsoft today announced that it intends to acquire DATAllegro, provider of breakthrough data warehouse appliances. The acquisition will extend the capabilities of Microsoft’s mission-critical data platform, making it easier and more cost effective for customers of all sizes to manage and glean insight from the ever expanding amount of data generated by and for businesses, employees and consumers.
-- Press Release "Microsoft to Acquire DATAllegro"

This is very interesting given the progress that Microsoft has made with its analytic services binding MS Office and SQL Server. Further quoting from the press release:

“Microsoft SQL Server 2008 delivers enterprise-class capabilities in business intelligence and data warehousing and the addition of the DATAllegro team and their technology will take our data platform to the highest scale of data warehousing.”
-- Ted Kummert, corporate vice president of the Data and Storage Platform Division at Microsoft

The direction for DATAllegro's data warehouse appliance is also made clear in the press release:

“DATAllegro's integration with SQL Server is the opti mal next generation solution and the acquisition by Microsoft is a great conclusion for the company.”
-- Lisa Lambert, Intel Capital managing director, Software and Solutions Group.

For those who don't know, DATAllegro is a data warehousing appliance company that utilizes "EMC® storage, Dell™ servers, Cisco® InfiniBand switches, Intel® multi-core CPUs and the Ingres® open source database".

So, whither Ingres in this acquisition? As we've written before here, Ingres is one of the earliest and strongest RDBMS products, which was absorbed by CA and then spun off again with an open source play in 2005. MS SQL Server, of course, started out as a rebranding of Sybase SQL*Server, until the partnership dissolved in the mid-1990's. Since then, MS SQL Server has been geared mostly as a workgroup and data mart server. It seems that a switch from Ingres to MS SQL Server could heavily undermine DATAllegro's business. In addition, the switchover in code to T-SQL will be a nightmare for developers. Add to that the challenges of moving from Linux to MS Windows, and from C/C++ to C# and it will take quite some time in production environments to iron out all the wrinkles.

In addition, while most seem to think that this puts Microsoft in a good position to challenge Oracle for the Enterprise Data Warehouse lead, it actually puts Microsoft directly into competition with other DW appliance vendors, such as Teradata. I truly doubt that this move will position Microsoft strongly into competition with either Oracle or Teradata, but merely marks another tactical error in Microsoft's increasingly desperate acquisition strategy to move deeper into the Enterprise on one hand, while striving to move further into the online space on the other.

More can be read at:

BI for iPhone

With the opening of the Apple iPhone AppStore on iTunes and with the iPhone2.0 software, I decided to take a look for iPhone BI apps.

The first, from Pentaho, is open source. "Pentaho's BI extension for iPhone works with Pentaho Open BI Suite 1.7. Download and configuration instructions, as well as a short, recorded video demonstration are available..." from Pentaho's iPhone page. Matt Casters has more on his blog, "pentaho and the iphone".

The second is not open source, but is free from iTunes, but "Requires the licensing of Oracle Business Intelligence Suite, Enterprise Edition Plus, and Oracle Business Intelligence Applications, Fusion Edition..." Wow. &#59;)

Even though I have the latest iTunes7.7, it's showing that my iPhone1.1.4 is up to date. We'll have to start playing with these as soon as iPhone2.0 is generally available - later today maybe.

SOAP vs REST and OSBI News

I recently joined Twitter. I must share the following:

Roebot: #e20 note to organizers: If your panelists do NOT know what SOAP and REST are they prolly shouldn't be on a mashup panel!!! WTF!! about 5 hours ago from twhirlend quotation
-- Aaron Roe Fulkerson on Twitter

To which I responded:

Joseph_di_P: @Roebot wiki(SOAP) is what you use in tub to get clean; wiki(REST) is what you do in tub when not using SOAP :-D Easy, yah! about 1 hour ago from Hahlo in reply to Roebotend quotation
-- my response on Twitter

I know, I know, all the important stuff happening in the Open Source BI related world this week, and this is what I blog about. Is it a sign of dementia when you crack yourself up? :)) :crazy:

Here's some of the more important stuff that's been happening:

There's much else to do, including some additions to our linkblog with open source for MDM and more open source communities. But, I'm tweeting. :D

Making Sense of Open Source

The highlight of JavaOne for me has become supper with Gianugo Rabellino, the founder and CEO of SourceSense. For now, each year…

"'The time has come,' the Walrus said, 'To talk of many things: Of shoes—and ships—and sealing-wax— Of cabbages—and kings— And why the sea is boiling hot— And whether pigs have wings.'"
-- from the poem "The Walrus and the Carpenter" within Through the Looking Glass by Lewis Carrollend quotation

… For our conversations are wide ranging and thoroughly engaging as we indulge our enjoyment of fine food and open source.

This year, Gianugo has been in the USA several times, for the Open Source Think Tank and the Open Source Business Conference, and on business. An employee of SourceSense even had a gig in the town in which I was raised. Unfortunately discovering it to be the arm pit of the United States. Ah well, it's why I live in the SF Bay Area now. &#59;)

We talked of tempering chocolate, unusual eating habits of various cultures, the worsening economy, European vs USA political views, and Gianugo even had me read, while sober, the Pronunciation Poem. [I flubbed a few words, and disagree with some of the rhymes given that my dialect is as Philly as a Chesse Steak.]

But we also talked about Open Source; all the open source related conferences and blogging and work and newsworthy activities of the past few months. Here are some of my thoughts.

Open Source is a Philosophy

This one has been bubbling around in my head for years. Open Source, in and of itself is NOT

  • a licensing template
  • a business model
  • a development methodology

It is a philosophy that can provide the framework for those three things with which it is most often identified. The variations among what the open source philosophy means to each of its followers can best be seen from the proliferation of licenses, business models and methodologies all claiming to be open source.

To me, the open source philosophy is very simple, and it can be applied to solutions for some very complex problems and concerns: the source is available to anyone who obtains the end-product, whether one has obtained the end-product through a no-cost download or through a purchase agreement of whatever type. The "source" may be the "source code" for software, but, to me, it should be whatever specifications and design documentation are required to recreate the end-product in either the original or a modified form.

There are a variety of reasons that a developer, an engineer, an inventor, a creator, might what to subscribe to an open source philosophy. And each individual or business must decide if those reasons make creative and economic sense for them. Once one has decided to subscribe to open source, or any philosophy, then all your other decisions will only lead to success if they are logically consistent within that philosophy. The licensing language, business model, economic forecasting, internal processes and external relationships should form a coherent whole within the underlying philosophical framework.

Business Models Will be Different for Different Markets

Organizations often look for the silver bullet or the golden mantra or the platinum ring that will solve their problems, lead to dominance in the marketplace, or allow them to rule them all. &#59;) So we hear a lot of talk about the "best" open source business model. This search ignores that fact that there are open source solutions for many markets. Without going into specific verticals, let's just consider four general, horizontal markets that are addressed by open source software.

  1. Information Technology Infrastructure: including operating systems such as the various flavours of linux and BSD unix, application servers, and other middleware such as the Mule ESB, WSO2 SOA solutions & KETTLE for ETL, web servers, email MTA, the Funambol Mobile Server platform, and many more.
  2. End User Applications on the Desktop/Laptop/Mobile-Device: with OpenOffice.org and its MacOSX offshoot NeoOffice, Projivity OpenProj, and Mozilla Firefox and Thunderbird likely being the most well known, but also including many thousands of open source projects for artistic creation and enjoyment, as well as office and personal productivity, and gaming, and all things that one personally does on a computer
  3. Software Development Tools: from the Eclipse IDE to hundreds and thousands of specific language libraries and everything else a developer might need; this is even more "by geeks for geeks" than the infrastructure area
  4. Enterprise Applications: these are end-user facing applications, often fulfilling mission critical needs, such as ERP, CRM and BI, and is the least mature market for open source providers and the hardest area to gain acceptance

No one business model, no matter how generically expressed, could satisfy these four disparate markets. Monetizing these areas will come from combining innovative and traditional packaging of support, customizations, system integration, training, licensing, and subscriptions.

Automated maintenance, repair and update networks might work very well for monetizing IT infrastructure, but might be insufficient for the other markets. Ad based monetization, directly or through partnerships with ad networks like Google's, might work for some end-user applications. Putting out the begging bowl, asking for PayPal contributions might also work. But for many open source projects, there simply isn't any path to monetization.

One area that I think has been insufficiently explored, and might well be the only path to success for the Enterprise Application open source vendors is Software as a Service. The SaaS approach, whether through partner channels or directly, is the most sensible means of monetizing a wide variety of open source applications. Embrace the open source philosophy, leverage the strengths of flexibility and community in the licensing, business models and processes, and monetize through SaaS delivery into vertical and niche markets.

And for the rest, just acknowledging that your company is a software company embracing an open source philosophy, and building appropriate support and licensing structures will be the best path.

Communities are Strongest when Open

Developing a community around a product is not unique to the open source world. Microsoft, Oracle, SAP, SAS, SPSS, IBM, etc, etc, etc have all developed and supported great communities made up of ISVs, VARs and Users. The difference however, is that open source projects are very dependent on their community. Conversely, communities are strongest when there are no artificial limits to communication.

I think that most companies are embracing at least this one ramification of the open source philosophy. Though it seems to me that when the source isn't open, the community can only converse and support each other in surface issues.

One area though, where open source vendors still seem to be uncertain, is the definition of community, and what groups make up their community, and if they must develop different communities for different categories of members: developers, users, non-software contributors, paying customers, or partners. I think this is self-defeating. There is only one community for each project: those who want to be involved and will benefit from that involvement, ultimately to the betterment of the project, whether directly or indirectly, whether monetary or not, whether contributing code or documentation or use cases or testing or playing around with the project or complaining about it or just soaking up the ambience. Can any open source project community manager give a valid reason why any of these should be excluded from the community or ignored?

Open Source Sales are Very Foreign to Most IT Buyers

Whether we're talking about senior IT management, the CIO, or the purchasing department, the idea of freely downloading the fully operational, unlimited version of a product, and using it for research, prototyping or production, without handholding, cosseting or sales incentives is complete anathema to those making purchasing decisions.

Did you ever see a "vendor bake-off" among products from Oracle, IBM and SAP without vendor involvement, without vendor sales engineers fine-tuning and providing weeks of free labour? Is there any open source company that provides a corporate jet and box seats at the super bowl? And is there anyone who has ever been part of selling (internally or as a vendor) a large, big-budget enterprise project to a CIO or purchasing committee, who doesn't believe that such freebies and perks are essential elements to the approval process?

Many, too many, open source vendor CEOs believe that bloatware (more features than most users will use), vendor lock-in and lack of interoperability, and high prices of initial licenses are the problems CIOs worry and that open source software solves. They think success will come from "80% of the capabilities for 20% of the cost" of their closed source competitors. I don't believe any of it. CIOs worry about being beat up by the business side and greasing the squeaky wheel. The two highest costs in a data center are people and energy, not licenses or maintenance agreements. Implementation costs: hardware, software licenses, personnel and training, are spread out over three to five years with appropriate tax implications. For really large projects such as a full ERP or data warehouse, there is at least a ten year life expectancy. Why do you think that there are so many mainframes and COBOL applications still around? The implementation costs were written-off long ago, and the on-going support costs are minimal because they still JUST WORK. Those with an eye to economics are more concerned about the total cost of ownership over the complete 10-year lifecycle of a capital project, than with the 10% or less of those costs that go towards initial software licenses.

So, if the only way for open source vendors to become big players, or as one CIO put it, to "move out of the junior varsity", is to look like the big players, to provide pre-sales engineering and collateral beyond a T-shirt, then what will this do the business model and the belief in "20% of the cost"? It will destroy it. This is why there are more openings for sales & marketing than for engineers at open source companies today.

Open Source Vendors will grow and evolve and come to operate more and more like the big players they're destined to become. As long as they continue to adhere to their form of the open source philosophy, are internally consistent and true to the logic of their open source framework, they'll continue to benefit from the true value of open source: flexibility and community knowledge to a greater degree than can be achieved by any closed-source vendor or any isolated, enterprise data center with home-grown solutions. Speaking of which…

Open Source is the Middling Ground between Build vs Buy

This is another point I've been hammering for years. Traditionally, IT shops were either build or buy. The build shops were very development oriented and created custom solutions to implement business processes and support business users. The buy shops bought COTS software, and either convinced the business that the software they bought implemented industry best practices, or spent millions of dollars in customization.

Build shops suffer from isolation and the total burden of maintaining and updating the software they built. When the only developer left who remembers and understands the code, retires, faith often becomes the best practice for ongoing support. :>>

Buy shops suffer from ongoing dependancy on the vendor(s) and ongoing compromises for the users.

Build shops should be (but often aren't) more flexible to respond to changing business needs, and can be more valuable to the business.

Buy shops can be more reliable and cost-effective - really. But not always.

One result of Y2K [remember that?] was that most IT departments became much more of a blend of build and buy, with buy decisions winning out. Coupling this with the facilitation of distributed workgroups made possible by the Internet expanding world wide, and offsourcing [outsourcing operations to offshore companies] decimated many corporate data centers by solving the personnel and energy cost problems. Guess what? Implementation and licensing and maintenance costs remain the same. Some CIOs use offsourcing as the reason that they don't use open source; the outsourced vendor isn't familiar with it, and has no incentive to use it.

As companies bring IT back inside, and as its importance to the business is once again realized, open source offers a third path to the traditional build or buy. However, offsourcing will still be a substantial part of the solution, and should lead OSVs to recognize the importance of offsourcing vendors and embrace them as partners and very important channels.

The IT shop that responds to the business using open source can be both flexible and well-supported. The open source vendors that make such IT shops succeed through flexibility and reliability will be the most successful ones. This can be achieved through the OSVs growing their support, professional services and training organizations, or by partnering with all sizes of PS, VAR and outsourcing firms, or both.


The real conclusion is that bringing the open source philosophy to fruition in business is still an evolving process. The advantages gained by the openness of the source and the strength of the community is being recognized by both the IT shops and the OSVs. While there are cost savings to be had, OSVs need to stop relying on initial licensing cost reductions as their main selling point, and begin to market the advantages to the IT shop of using open source: responsiveness to changing business needs and increasing reliability over time, all while providing the best TCO and ROI.

There's a lot more to be said on all of these topics and opinions, and maybe I'll even get the time do so. :p

Startup Camp Difference

Startup Camp is a mix of traditional conference panels and an open space "unconference".

The juxtaposition of the two is a bit jarring in some ways. 'Tis somewhat like going from a warm fuzzy blanket [panels] to an exhilarating plunge into the chilly North Pacific [unconference sessions]. Actually, panels have been losing their appeal over the past few years. A good panel is a lively debate. The panelists engage each other. A good panel is both entertaining and informative. Too many panels today bring to mind an event from my college days. This is an unfair analogy but I'll tell the story anyway. &#59;)

The only diner within walking distance of the college was the S&W [or, as it was traditionally known, the "Slut & Whore" - hey, don't yell at me; it describes the area well. Think of the Tenderloin in SF on a bad night.] Anyway, after an all-night study session, or maybe a drunken game of Diplomacy, we made a visit to the S&W in the wee hours for some coffee and rice pudding. Among the other patrons were two old transients [a.k.a. hobos, stumblebums]. They were at opposite ends of the diner, but seemed to be having a loud argument on who was the greatest baseball player of all time. After a while, we realized that they weren't arguing. They were merely proclaiming in loud voices on the same subject. OK, the panels aren't that bad, but getting close.

Another problem with panels today is that most panelists blog. If you are at all interested in the subject, it's likely that you read their blogs and already know their opinions.

Ah well, onto the good part.

The unconference sessions are intimate and, as mentioned before, exhilarating.

The only problem is that there are so many great topics being discussed at the same time, that it's like being in a candy store: you don't what to grab onto, there are just too many choices.

But you can read all about what happened on the Startup Camp wiki, if the attendees upload their notes. :D Just follow the links from the Startup Camp Unconference schedule at the bottom of the page.

Tomorow will be running back and forth between Startup Camp and CommunityOne. Can't wait.

Startup Camp 2008 Sunday Morning

Today, I'm at Startup Camp, in conjunction with CommunityOne, which starts tomorrow, in conjunction with JavaOne. The schedule is online, but this is a camp, an unconference, and the rules are different.

The introduction is over, and the keynote with Jonathan Schwartz and Om Malik is going on now.

A question from the audience, essentially that there is no simple, online solution for a Java developer to just go online and develop, led to an interesting side discussion where I'm seated about what it means to be a developer. The old folk at the table, have a much broader definition of a developer, and the skills they should have, than the young'uns, who are more focused on just one language, and just code monkey banging away.

The first panel is starting up, moderated by S. Neil Vineberg, President, Vineberg Communications. The topic is branding, and how the brand comes out of the founders and the culture that they create.

One interesting point for me, is that while the panel is discussing the importance of branding, and that the brand can flow from the founders' personality and the culture, I have found that if the founders are too focused on creating a brand, and selecting their brand category, they'll fail to infuse their brand with their personality, that is, the brand can seem sterile and contrived. Branding is very important, but, to me, it must come naturally from the company if it is to truly reflect the company. Of course, sometimes you have a founder that you need to keep locked in a closet, feed caffeinated drinks and pizza, and never, ever let them talk to the customers. :>>

The use of social media is a given today, especially for startups. The use of Ning [why doesn't PeopleAggregator ever get a mention?] and the growth of social network platforms for startups [others] to grow their own social networks is a great indicator of this.

For all the talk of social media, the message of this panel seems to be that traditional methods, through PR, is still the best way to reach out and get your message across; especially outside of the technology centers like the SF Bay Area. However, it seems to me that when the panel starts talking about what's really effective, they use terms like "community" and "authentic conversations". It also seems that creating markets and driving markets is the better than pushing a product, in terms of success, and in terms of getting noticed.

The morning is done. I'm looking forward to networking and learning throughout today and tomorrow, to CommunityOne and seeing Michael Coté at Redmonk's Unconference, and a great conversation with Gianugo Rabellino at JavaOne and for supper on Thursday.

OSBC2008 Presentations Downloads

InfoWorld has made downloads available for selected presentations from this year's Open Source Business Conference. The links will take you to PDF files. But, Matt, where are your video mashups?

MuleCon2008 Closing Campground

The community has been brought together for the final campground session. Rather than sing kumbaya, we've been teased with t-shirts, toys and books from O'Reilly and being treated to a demo of a hot deployment by Travis Carlson. 'Tis all drag-and-drop goodness, with dragging jar files (bundles) as connectors and apps from a MacOSX Finder into Mule and watching the results in Terminal.app. Ohhh! Ahhh! Safaris shows Mule saying "Hello Travis, welcome to MuleCon" B) OK, I'll admit that this is pretty neat and should satisfy the most addicted user of Tibco wizards. This won't appear in Mule Enterprise Edition until after the 2008Q3 release of Mule EE 2.0.

One little aside… I was talking to an ex-TIBCO employee who said that as old as TIBCO is (grew out of a Teknecron business that was founded in 1985 and became an independent business in 1997), the user even only draws about 400 people. This is only the second MuleCon [there was a sort-of pre-MuleCon in 2006 at a bar in London, but let's not count it), which last year drew around 100 folk and this year brought in around 240. That's phenomenal growth and shows the excitement that good software can bring to its users.

Questions for the Campground from Day 1 and some new ones.

  1. AMQP is an emerging messaging protocol and is platform independent. To answer the question about dot-Net integration with Mule, this is part of the answer.
  2. BPEL & BPM? Workflow tools, such as JBPM, that don't depend on simple web services, work better with Mule. Travis has done some work with this including the JBPM plugin that is in the current distribution of Mule, and there is more on the Forge. Mule also works well with Oracle and Iona's solutions. Mule Services and Mule Events are complementary.
  3. Mule is moving to Spring2, and has MuleSource thought about the exciting things that can be done with AOP in Spring2? Not so much yet, but it is among the next thing to look at. Mule Galaxy might be one of the first areas where MuleSource would leverage AOP to create runtime governance.
  4. Can Mule Galaxy can be used to manage other artifacts? The short answer is yes.
  5. How are folk clustering Mule? The typical way is to leverage an existing clustered app server. They are seeing more where the ESB has a state itself. They're looking at things like TerraCotta for such use cases.
  6. Discussions of rules engines and routing, but much work to be done, hopefully through the MuleForge.

Dave gave some closing remarks and that's all 'til next year.

January 2019
Mon Tue Wed Thu Fri Sat Sun
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31      
 << <   > >>

At the beginning, The Open Source Solutions Blog was a companion to the Open Source Solutions for Business Intelligence Research Project, and book. But back in 2005, we couldn't find a publisher. As Apache Hadoop and its family of open source projects proliferated, and in many ways, took over the OSS data management and analytics world, our interests became more focused on streaming data management and analytics for IoT, the architecture for people, processes and technology required to bring value from the IoT through Sensor Analytics Ecosystems, and the maturity model organizations will need to follow to achieve SAEIoT success. OSS is very important in this world too, for DMA, API and community development.

37.652951177164 -122.490877706959


  XML Feeds