Category: "Computers and Internet"

Complex Data Visualization at MySQL BI DW BoaF

Just got home from the MySQL Data Warehousing and BI Birds of a Feather gathering (BoaF). I'm tired, but my mind is on overdrive. 'Tis a great feeling.

First, I want to thank Lance Walter of Pentaho for introducing Clarise and me to the group as publishers of the OSBI Lens on Squidoo, this blog and the OSS wiki.

Clarise and I had a great conversation with Dr. Jacob Nikom of the MIT Lincoln Laboratory. The conversation ranged from the great Chimay beer that Matt Casters of Pentaho and Lead Architect of KETTLE, brought with him from Belgium, to

  • Data Modeling and Relational Algebra/Theory: the integrity of the model
  • Bayesian Statistics, Weibull Analysis and Tensor Calculus for mathematical modeling of complex systems [I love it when n-dimensional eigenvalues start floating in front of my eyes]
  • Meeting the needs of different types of users: managers, scientists, business folk
  • supplementing historical data warehouses with [near] real time data using ESB and dashboards
  • data visualization of complex data sets such that the analyses and limitations can be grasped at different levels by different users
  • collaboration among distributed workgroups of disparate career backgrounds and cultural pre-dispositions
  • use of Second Life and other virtual worlds for collaboration and data visualization
  • a calculator is to a computer [think if statment] as a flat file is to a relational database [think where clause]
  • early USSR vs. British knock-offs of IBM mainframes
  • Complexity as the balance of robustness and fragility

At various times in this discussion, we were joined by Sherman Wood, Director of BI at JasperSoft, and one of the legendary Mondrian developers, Julian Hyde of Pentaho and Mondrian Lead Architect, and Nicholas Goodman Director of BI for Pentaho.

And if you put Nick and dashboards and virtual worlds in the same post, then you have to mention Discoverer meets Duke Nukem.

Jakob, et al, thank you so very much for a great conversation.

BI and DW at MySQL 2007

Here's a guide to what's happening in open source business intelligence and data warehousing at the 2007 MySQL Conference and Exposition.


  • Fabrice Bonan is co-founder and Chief Operating Officer of Talend.
  • Roland Bouman has been awarded the MySQL 2005 Beta Challenge, and the MySQL 2005 Forum Leader of the year.
  • Matt Casters is Pentaho's Chief Architect for Data Integration and the founder of the KETTLE open source data integration tool.
  • James Dixon is Pentaho's CTO and Chief Geek.
  • Virgil Dodson is a Technology Evangelist at Actuate Corporation.
  • Victoria Eastwood is a consultant with Infobright Inc.
  • Ian Fyfe is currently director of product management and product marketing at JasperSoft Corporation.
  • Julian Hyde is Pentaho's Chief Architect for OLAP, and the founder and lead for the Mondrian OLAP project.
  • Barry Klawans is responsible for the technical vision and direction of JasperSoft Corporation's products.
  • Brian Miezejewski is a Principal consultant for MySQL.
  • Lance Walter is Pentaho's VP of marketing.


Exhibit Hall

The Exhibit Hall Hours are:

  • Tuesday, April 24 - 10:00 a.m. - 4:30 p.m.
  • Tuesday Night Reception - 6:00 p.m. - 7:30 p.m.
  • Wednesday, April 25 - 10:00 a.m. - 5:00 p.m.

OSBI related companies with booths at MySQL 2007 include Actuate Corporation, JasperSoft in Booth #409, Pentaho, and Talend in Booth #606.

And don't miss the Birds of a Feather gathering (BoaF): MySQL Data Warehousing and BI with Roland Bouman, Matt Casters, James Dixon, Julian Hyde, Brian Miezejewski, Lance Walter, and Nicholas Goodman. The related sessions on GIS, scalability, performance and tuning, clusters, and The Declarative Power of Views, all look very interesting.

See you there.

Cubulus Presentation

We discovered a new OLAP engine, Cubulus, registered on Sourceforge on 2007-03-07. The developer has an interesting presentation on the web, as well. [Just click anywhere on the page that isn't a link to somewhere else, and you'll go to the next slide. Or was that obvious to everyone but me?] Even though the project is pre-alpha, we've added it to our linkblog [look in the side column under OLAP]. The first database supported is MySQL, but they've recently added "hacks" for PostgreSQL support as well.

JasperSoft at MySQL2007

JasperSoft will have a booth and be presenting on three topics at the MySQL Conference & Expo 2007.

2007 MySQL Conference & Expo
April 23 - 25, 2007
Santa Clara Convention Center
Santa Clara, California
Booth #409 at the Exhibitors' Hall

2007 MySQL Conference & Expo
April 24, 2007
3:35 PM - 4:35 PM
Santa Clara Convention Center
Santa Clara, California
Using the JasperSoft BI Suite for Data Integration, Reporting, and Analysis with MySQL / Barry Klawans

2007 MySQL Conference & Expo
April 25, 2007
1:40 PM - 2:25 PM
Santa Clara Convention Center
Santa Clara, California
Integrating MySQL and JasperReports / Barry Klawans

2007 MySQL Conference & Expo
April 26, 2007
2:30 PM - 3:15 PM
Santa Clara Convention Center
Santa Clara, California
Embedded JasperSoft for MySQL Inside Enterasys / Barry Klawans

In addition, JasperSoft is a Platinum sponsor of this year's conference. For other upcoming events in which JasperSoft is a participant, visit their events page.

Reporting for OpenOffice.ORG

As we cite in our OSBI Daily, an integrated reporting solution is coming to as reported by Pentaho and Sun Microsystems today, planned to be incorporated into the next major release of OpenOffice.ORG, version 2.3.

The jFreeReport open source report project was brought into Pentaho in late 2005, and has been enhanced with "drag-and-drop report designer, MDX support, and integration with the Pentaho platform for report scheduling, security, and portal integration[;] Pentaho's recently-announced standards-based metadata layer and AJAX-based ad hoc query tool will also support and extend Pentaho Reporting's capabilities" allowing users "to create reports with content from the Base database as well as a wide range of proprietary and open source relational databases, OLAP and XML sources".end quotation
and read more on GullFOSS

I see this as a very powerful development for Open Source adoption. This shows the power of

  1. the open document format (ODF) and open standards in general
  2. interoperability among open source projects and solutions, and
  3. alliances among open source companies and supporters.

This is one more example of how open source business intelligence solutions and open source desktop applications can give established proprietary vendors a run for the money. While many open source commercial organizations aren't yet comfortable with competing head-to-head with large enterprise vendors and like to use the mantra "80% of the capabilities at 20% of the cost", I think that the open source platforms, appliances and solutions coming from the likes of JasperSoft, Pentaho, SpagoBI, Greenplum, Ingres, Mulesource, Alfresco and Sun Microsystems prove that open source can compete for large-enterprise, government, university and SMB customers with their combination of the advantages of community, flexibility and standardization, powerful and extensive functions, and amazing cost effectiveness.

Of course, being Apple users, we have to hope that this solution will quickly find its way into NeoOffice, once it's proven out in 2.3. &#59;)

BI Team Blog on OSBI

Welcome to the wonderful world of OSBI [not Oklahoma State Bureau of Investigation] &#59;) Our linkblog now has over 60 links to OSBI projects, and other sites. I hope that you find it useful. We'll be updating our wiki and lens on Squidoo to match it.

I just ran across the BI Team Blog from Face and Hannibal, as they "uncover open source business intelligence", discovered via Technorati. I tried to leave a comment, but I'm having this little problem.

At any rate, I've added the BI Team to our linkblog. I plan to follow along as they check out Pentaho and JasperSoft BI Solutions. Check them out.

Upadate 20070415;13h05: I had set this post up as a trackback to the BI Team, and had recieved an "OK" back from the server, but I see it hasn't appeared 17 hours later. /sigh Let's try again... Hmm, says that they already have a ping from us for that post.

An Afternoon at JasperSoft

Yesterday I spent the afternoon with Andrew Lampitt, Senior Manager Business Development, Nick Halsey, VP of Marketing, and Ian Frey, Director of Product Management & Product Marketing, at JasperSoft HQ in downtown San Francisco.

We discussed a wide variety of topics such as all of our backgrounds, JasperSoft's history, strategies, and future plans. Here's a sampling of the conversation.

  • As announced in January, JasperETL is based upon Talend Open Studio. JasperSoft found Talend Open Studio to be a very mature and well-planned data integration product. While the Talend Open Studio ETL tool was only released in the latter part of 2006, it is the result of a three year R & D effort led by former Informatica and DataStage personnel system integrators [updated 20070416]. The goal of JasperETL is to provide an easy-to-use but fully featured graphical ETL tool to facilitate data integration for the Jasper BI Suite.
  • There are over l30 active projects available for download on JasperForge. According to the forge statistics, 38 are public and 93 are private. Some of these projects use JasperSoft products, some extend them, and some provide embedding or integration of JasperSoft products into other applications. This led to a discussion of the Jasper4 program vs. other adapters; JasperSoft provides the functions in a Jasper4 branded application, such as Jasper4salesforce, while adapters or other applications from the community or third-parties would not carry the Jasper4 brand. CRM is a particularly active area as exemplified by the SugarCRM adapter, the partnership with Centric [see the OSA announcement] & the previously mentioned Jasper4salesforce.
  • JasperSoft has 5000 payinq customers in 81 countries, approximately half are ISVs embedding JasperSoft capabilities into their own products, projects or offerings.
  • JasperAnalysis & JasperServer are separate projects but share the same framework, which is why they were originally released on the JasperForge as JasperIntelligence, but they are currently being branded as Jasper BI suite; Ian gave a presentation showing the architecture and roadmap.

Overall it was an informative afternoon, and we're looking forward to working more with JasperSoft BI Suite in the future.

Cotinuous Process and Code Improvement

We're constantly recreating our 6D™ project management methodology. It started with combining Clarise's software development and project management experience with my aerospace system engineering and program management experience to adopt strict project controls to modern business needs for responsive software development and system integration processes working through distributed personnel. Well, here's a quick thought... software development and deployment should move away from traditional release cycle concepts to one of continuous process/code improvement within SaaS and virtual appliance environments. No code is alpha nor beta nor production, but a continuum of changes and adaptations responding to fluctuating business needs; done within a well managed environment to prevent security errors, poor performance, "garbage out" and junk code. So as we're assuring that our 6D™ [six dimensions of a project] is in accord with the PMBoK, we'll be keeping this thought in mind as well, and let's think beyond Extreme and Agile programming and continuous process improvement for software quality.

MuleCon2007 Looking-back

Our main interest in ESB is to supplement ETL, and inject real-time data into dashboards, OLAP, maybe even reports. BPEL/BPM as an offshoot of BI is also of interest. ServiceMix/Tuscany don't seem to be as useful in this regard. My guess is that ServiceMix/Tuscany will, as most Apache projects, be relegated to enhancing web servers, but not extend into other areas of the enterprise. Of course, everything might be pouring out of web servers in the future.

MuleCon was interesting mostly because of the enthusiasm of the customers. The week before we went to the Yahoo! sponsored open source content management system symposium [mostly to learn more about Alfresco, and to see the next generation of Joomla!]. That was pretty much all geek, all the time - very informative, but focused more on the technology, than the uses of the technology. MuleCon was smaller, and evenly distributed between presentations by customers and presentation by Mule core engineers. Not an hour went by without the phrase "use case" being bantered about. It was really very well done, despite some failure in conference organization. [It ended without a conclusion, just sort of petered out, with some folk in an unorganized discussion of the Scripps challenge, and others just milling about, wondering if the conference was over.]

People had come from all over the world, both to present and to participate; just over 100 were there for training on Tuesday, and about 150 were there on Wednesday. There were few Mule experienced people in the audience. Some had EAI or SOA/ESB experience [mostly Tibco] and were interested in Mule as an alternative, both for flexibility of adapting code, and for cost savings . Of course, the folk most willing to talk and to exchange business cards were other consultants. ;-) One interesting point in the side conversations was on the subject of configuration: some wanted wizards, others felt the XML files gave better control and facilitated understanding of the underlying process.

MuleSource seems very focused on their customers, and on providing business solutions through a flexible ESB without underlying infrastructure/technology assumptions. They should consider more such events, in varying locales. Hopefully, they learned as much about their potential users, as the participants learned about Mule.

We're blogged from MuleCon2007 throughout, and very much enjoyed the experience.

MuleCon2007 Selection

Scripps [TV] Networks [one of their properties is the Food Network] on SOA and ESB Selection. Starting with a design, going from a top-down approach, they began their selection. The next step was to run a few pilot projects to test a bottoms-up approach using three use cases.

  1. Asset Ingest, assets being videos, closed caption files, and the like
  2. Enterprise Asset Notification System, using a publish and subscribe model
  3. Asset Retrieval

Asset Retrieval use case was detailed, and forms the basis of the design challenge. The design challenge to the audience is to go to the whiteboards and work it out. A true exercise of an open source community. :p

  1. Performance is the primary criteria for success.
  2. Assets are registered with a Handler.
  3. Handlers are Spring configuration beans.
  4. Dynamic Handler Data plus Static URL's equals Asset Retrieval ReSTful URI.

Ross, Andrew, Eugene and Travis will lead the teams.

We're blogging from MuleCon2007 throughout today and tomorrow.

July 2020
Mon Tue Wed Thu Fri Sat Sun
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31    
 << <   > >>

At the beginning, The Open Source Solutions Blog was a companion to the Open Source Solutions for Business Intelligence Research Project, and book. But back in 2005, we couldn't find a publisher. As Apache Hadoop and its family of open source projects proliferated, and in many ways, took over the OSS data management and analytics world, our interests became more focused on streaming data management and analytics for IoT, the architecture for people, processes and technology required to bring value from the IoT through Sensor Analytics Ecosystems, and the maturity model organizations will need to follow to achieve SAEIoT success. OSS is very important in this world too, for DMA, API and community development.

37.652951177164 -122.490877706959


  XML Feeds