Category: "Computers and Internet"


Today I had the good fortune of speaking with Doug Moran, Founder and VP of Community, and James Dixon, Founder and Chief Geek/CTO of Pentaho about their OpenScrum methodology. Doug had responded to my question on LinkedIN, "What has been your experience with Master Data Management and SOA, SaaS and Agile, in any combination?"

In creating the OpenScrum Agile software development method, Pentaho faced many of the same challenges that we've been facing with some of our customers:

  1. Extending Agile methods to be used by distributed workgroups, in Pentaho's case this includes every inhabited time zone on this planet for their extended community and several time zones in the USA and EC for the core group
  2. Adjusting Agile methods to work with several products or modules or projects in parallel
  3. Documenting the communication among team members even when "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation" isn't possible - quote taken from the Agile Manifesto Principles
  4. Dealing with the concepts of timeboxes and rhythm
  5. Deciding on who best fills the roles of product owner, scrum master, and the various levels of commitment among the core group, the extended team and the community at large
  6. How best to involve QA/Test - a subject on which Scrum gives little guidance

The OpenScrum methodology deals with much of this, as does our ever evolving N Dimensions of a Project methodology, which has gone from 5D™ in the mid- to late-90's to the 8D™ iteration that we're currently documenting.

The most interesting point to come from our discussion is that Doug and James have come to many of the same conclusions and principles as have Clarise and I, even though our "experimental sample" &#59;) is the Pentaho open source communities on the one hand, and the various internal IT and SaaS software development distributed workgroups on the other.

One of the first things we discussed concerned the idea of fixed timeboxes and rhythm. The idea of rhythm is very attractive, but very difficult to achieve in most situations. In many ways, the concepts that Agile methods address: responsiveness to changing user needs and a changing market, coupled with the realities of changing personnel and other organizational and business changes, make achieving a short-term rhythm very difficult. As an organization matures, perhaps a longer term rhythm, such as the quarterly or "seasonal" rhythms of might be a realistic goal. As an organization first moves into an Agile process, and develops an Agile software development methodology that fits with their evolving culture and ecosystem, sprint timeboxes must be flexible, and achieving a fixed release rhythm isn't practical. For our part, we make setting the timebox part of each sprint planning meeting. Once set, the timebox is inviolable, and any further negotiations must be around the feature set to be achieved in the sprint.

In the same area of time, James and Doug shared their observation that the sprint burn-down chart can result in a false sense of security or panic. No work effort follows the nice smooth regression line of a sprint burn-down chart, but more sophisticated modeling, to plan for the stepped nature of real-life work, is beyond simple spreadsheets. It's not beyond the current math modeling state of the art nor the capabilities inherent to the various analytic and data mining modules of Pentaho. While Pentaho currently has no plans for a "Pentaho for OpenScrum" similar to "Pentaho for Jira", it's certainly food for thought for the Pentaho community, and led to some fun brainstorming on our call.

An issue related to time, and space in this instance, is that almost everyone today works in some form of distributed workgroup. This may include the occasional telecommuter, or, as in the case of Pentaho and other open source projects, communities that span the globe. This would seem to preclude most Agile methods, especially Scrum, which puts a great deal of importance on stand-up face-to-face meetings, specifically the daily scrum. One way to get around this is to use instant messaging or teleconferences, especially if remote whiteboarding can be used - though always one group or another is inconvenienced by the selected time. Another way is to use blogs, wikis or forums to supplement the daily scrum, especially for those who might not be able to attend due to the time selected. One other things that we've done is to record the daily scrum, either audio alone, or with video. The take-away here is that some form of asynchronous communication, and keeping an historical reference, is a necessary addition in the face of the flat world of software development today.

The idea of parallel scrums has been batted around within the Agile communities for some time now. The reality is that sprints must often be done in parallel and some individuals are going to be assigned roles in more than one sprint team simultaneously. The reasons for this are many, but the bottom line was discovered by project managers long ago: small teams with sharply focused goals are more often successful than not. Large, complex or broadly defined goals must be managed as programs or portfolios, not as projects. We've followed separate strategic, tactical and implementation tracks for over a decade, with a modified iterative waterfall approach cycling through these three tracks throughout time. This allows us to easily incorporate parallel implementation sprints for each tactical project. The OpenScrum methodology has some good depictions of their approach to parallel sprints. For another perspective, I can draw upon a buffet-line-conversation I had with a senior project manager [who felt that Agile was "crap"] at a recent SFBAC (San Francisco Bay Area Chapter) meeting of the PMI (Project Management Institute) and that is the fact that Agile Software Development methods are not generally applicable Project Management methods. [My opinion and not necessarily representative of anyone else mentioned in this post.] As such, it's much easier to see how parallel sprints can be planned. With Agile methods, we're developing software, not dams, bridges or space shuttles. Software development is much more of an art than an engineering discipline, and the Agile methods allow for this artistry.

The roles of Product Owner, Scrum Master and Scrum team are very important to Scrum and OpenScrum and 8D™ and still requires refinement. Many folk that we've encountered in our consulting, naturally assume that the role of Product Owner is best filled by an existing Product Manager and that the Scrum Master is a lead engineer. But just as your best technical person is often a poor choice for any management role [soapbox time: forcing great techies into management roles as the only path for advancement and "the big bucks" is just plain stupid today] the lead engineer may not be the best Scrum Master. We feel that the Product Owner should be a true representative from the user community, or at the very least a customer advocate. A Scrum Master must be skilled in corporate communication, escalation and, yes, politics, and empowered to clear obstacles that could prevent the sprint from achieving its goal. In addition to these two roles, the Scrum Team should have no more than seven (7) other [full time equivalent] individuals with technical and business skills, and subject matter expertise that assure the sprint's success. This is a guideline, with lots of ways to be implemented.

Another area of discussion that deserves weeks of attention, not the few minutes that we could devote to it in our call, is how best to incorporate QA and all the various testing efforts that any engineering effort requires. Well-thought-out test scenarios and automation certainly play an important part, but that's not the end of it. And we didn't get anywhere near the end in our discussion. Most Scrum articles, books, etc. kind-of ignore QA. Other Agile methods, such as Test Driven Development are centered around it.

We believe in incorporating QA and Test as part of the core group, and that code review & unit testing, at the least, must be a part of the automated daily build process. But what about regression and User Acceptance Testing (UAT)? If the idea is that each sprint results in usable code, isn't regression testing a major part of getting to usable code? Isn't UAT the proof of the pudding? But are such activities a part of the Sprint, a separate, overlapping/parallel Sprint, or a separate activity altogether? And what about QA of specifications, documentation, data integrity and metadata?

There are also some issues specific to open source companies/projects related to User Acceptance Testing, and user satisfaction metrics in general. It's one thing to count downloads, but how does one measure the comments, or really, the lack of comments in the forums? Open source companies often don't know who has downloaded their product, and don't whether the lack of comments on a new release is indicative of satisfaction or disinterest or even disgust. One way in which an open source company can get some amount of automated feedback is to have the product report an heartbeat, or other "phone home" technique [see Positive Feedback Enablers in the OpenScrum methodology]. Pentaho has implemented an "opt-in" or "opt-out" system for their heartbeat and has had no negative feedback for implementing this system.

As you can see… many questions. The answers are often dependent upon the supporting infrastructure, maturity and culture of the organization or community.

The culture results from the personalities of the pigs [the truly committed], the chickens [the involved], the sheep [who prefer to be herded] and the goats [who prefer to be led]; and don't forget the penguins, cats and lone wolves either. :>> Whether in an open source community or a corporate team, a manager may know the core group, but might only guess at the extended team or overall community. And, as with any generalization, the devil is in the details. This is why our methodology has gone from 5D™ to 8D™ over the years, and why James has developed both the Beekeeper and OpenScrum methodologies. From the initial framework for success, one must apply the principles of Agile even to the development of your Agile methods. Again and again, we see the need to adapt to specific situation, cultures and objectives.

We talked for well over an hour. One area that we didn't have time to explore was the Sprint Retrospective vs. Sprint Review. The OpenScrum methodology only discusses the Sprint Retrospective [among the truly committed] whereas we also allow for the Sprint Review [among all interested parties] in our "lessons learned" processes. Perhaps we can go into this further on another call or in the comments.

As always, talking with the Pentaho folk is lots of fun. We're always encouraged by their inventiveness in their execution as an open source company.

Wow Oracle and BEA Sun and MySQL

Oracle finally buying BEA Systems was expected. Once the Emerald City on Redwood Shores starts annexing a neighbor, it gets what it wants. A price has been reached and is being reported as 8.5 BILLION US dollars.

Sun Microsystems has announced that it is purchasing MySQL and will be competing with Oracle, IBM and Microsoft in the RDBMS market. Have I been living in a fog? Well, yes, since I live on the San Mateo Coast &#59;) but I hadn't heard any rumors about this at all. In Jonathan Schwartz's words Sun is "Helping Dolphins Fly". Some other opinions are more along the line "Extinguishing the LAMP". The purchase price is ONE BILLION US dollars, which, I think, makes MySQL the first open source company to break the billion dollar ceiling.

My own take on the former is that Oracle needs BEA to really make the transition to being the lynchpin for SOA and MDM. And since my career has been moving in this direction as well, I'm glad that it's finally happened. Though being an OrAlum myself, I do have sympathy :>> for some of the BEA folk with whom I've worked recently, and our HR consultant, who is also now a BEA, soon to be Oracle, employee.

And looking at it in that light, Sun's Glassfish and MySQL make a great open source platform for SOA and MDM. And since I've been much more focused on open source than closed the past three years, I'm really excited by this news.

Sun has been a great company since its inception. I've never been employed by them, but they have been a customer and partner over the years. And while I think that Scott McNealy is one of the all-time best corporate leaders, Jonathan Schwartz has brought Sun into the open source world with great strategic and tactical moves.

I think that watching Sun move MySQL forward in the Enterprise will make for a fantastic learning experience over the next few years.

These two moves may finally move SOA and MDM from hype through to mainstream adoption.

Hibernate Tools for Eclipse 3.1

I'm currently on a gig at a very interesting SaaS company. We're introducing and creating agile methods, creating a new SOA with MDM and recreating the applications in the new architecture. One snag that we hit is that the company is using WebLogic 9.2 and the most recent Hibernate Tools won't work with the WebLogic WorkShop, which is based on Eclipse 3.1.

Can anyone point me to an archive where I can find Hibernate Tools for Eclipse 3.1? The team can't find it anywhere. What's up Hibernate? Use the latest or forget it? XX(

On a related note, I want to give Kudos and a huge hoozah to Martin Ying, Principal Consultant with BEA Systems, Inc. He's accomplished an incredible amount in the four days he's been here. I heartily recommend Martin to anyone wanting to get started with developing in WebLogic Workshop. He's amazing. Thank you, Martin.

Bayon is Back

When I met with John Sichi back in August he let a cat out of the bag. When I was speaking to Nicholas Goodman in late September, he confirmed the cat, and now he's announced it.

Bayon Technologies is back, and Nick has gone from Pentaho employee to Pentaho partner. We at IASC welcome Nick back to the land of consulting and look forward to working with him in the future.

Actuate Launches BIRT Exchange

From a press release:


Developer Community Site Serves as a Dedicated Resource for Eclipse BIRT and the Actuate BIRT Product Line

Actuate today announced the launch of BIRT Exchange (, a dedicated resource for the Eclipse BIRT developer community and the Actuate BIRT product line. The growth of the Eclipse BIRT developer community and BIRT’s increased use in enterprise deployments has created demand for supported products and services based on Eclipse BIRT.

BIRT Exchange provides a dedicated site where developers can learn about support and services for Eclipse BIRT, download Actuate BIRT products and participate in the growing Eclipse BIRT ecosystem. Developers can share code and expertise with peers, helping to accelerate the flow of knowledge and information within the community.

Available on BIRT Exchange:

· Demos — a collection of self-running learning tools on everything from AJAX interactivity for BIRT reports, ad-hoc reporting with BIRT templates, spreadsheet reporting, using sub-reports, parameters and parallel reporting

· DevX — a dynamic content hub where BIRT developers can quickly find, share and discover useful BIRT-related technical information including code snippets, report designs, technical tips and tutorials. DevX makes it fast and easy for developers to search, upload, download and rate content items

· Documentation — official documentation for Eclipse BIRT and Actuate BIRT products

· Searchable forums — on Eclipse BIRT and the Actuate BIRT product line, including a mirror of the Eclipse BIRT newsgroup, which is also searchable

· Software downloads and detailed product information — for Eclipse BIRT and the Actuate BIRT product line, enabling rapid report deployment, spreadsheet reporting and ad-hoc end user reporting

· Support Resources — information on support, services and training available for Eclipse BIRT and the Actuate BIRT product line

· Wiki & Blog —facilitates an active dialogue among the BIRT developer community


· Virgil Dodson, Java Reporting Evangelist at Actuate Corporation –“Eclipse BIRT and the Actuate BIRT product line are a powerful combination of tools that let Java developers spend their time doing what they do best: creating innovative and impactful applications. BIRT Exchange gives developers a place to share and learn from peers about the latest and greatest technology and techniques for adding reporting to their applications, so that they can more easily meet their end-users’ reporting needs.”

· Mike Milinkovich, Executive Director of the Eclipse Foundation – “BIRT has been one of the most popular and successful projects at Eclipse since its initiation by Actuate in 2004. BIRT has successfully grown a large community of developers around it, and has helped extend the Eclipse ecosystem into enterprise reporting.”

· Mark Lynd, President, Firescope – “When we first used BIRT to embed reporting within the Firescope platform, we were under time pressure and needed to quickly come up to speed and to understand our different options. BIRT Exchange would have saved Firescope a lot of time and frustration and made life easier. This is going to be an invaluable resource for any BIRT developer.”


· Ask BIRT video

· BIRT 2.2 video with indexed menu

· Interview conducted by Michael Coté, RedMonk

· BIRT on Ohloh

· Actuate page

· Summary of Actuate’s 2007 Open Source Software Survey

· BIRT Exchange Screenshots

· Eclipse BIRT web site

· BIRT Exchange blogend quotation

John Sichi of LucidDB

Earlier today, I met John Sichi for coffee at the Half Moon Bay Coffee Co. in the Stone Pine Center. John is also a Coastsider and very involved in open source data management & analytics. We spoke of many things: our histories, folk we know in common such as Julian Hyde and Nicholas Goodman and some Oracle alums, happenings in the open source BI world, Pentaho, JasperSoft, SpagoBI, and lots of good story telling.

Mostly though, we spoke of LucidDB, LucidEra, & metadata management. I've been asked not to blog about some of things we discussed, so I'm just going to be safe and say that I am very impressed with what LucidEra is doing in BI SaaS. I'm also looking toward including the amazing capabilities of the column-store open source LucidDB in some engagement, somewhere, as soon as I can.

CT2007 W08 BI Strategy

We've completed presenting at Campus Technology 2007. Rather than a normal stack of slides in Keynote or PowerPoint, we presented our mind about six topics:

  1. What is BI?
  2. Roadmap
  3. Management
  4. User Involvement
  5. Open Source Solutions for BI
  6. Economics

Yep, rather than taking notes and brainstorming using mind maps, we presented using a mind map.

Our conclusion from preparing this presentation and from listening to our audience of around 20 folk, is that the key to a cost-effective strategy for BI is User Involvement and Open Source. More than just the open source solutions however, is what we learned from open source projects in managing the core and extended project teams, as well as all possible stakeholders as a community.

Update: Here's a PDF of our handout as well. As you can see, the handout was supplemental and background material for the mind map.

CT2007 Cost-Effective BI Handout

We would love to hear your thoughts, after traveling around our mindmap.

Alfresco Liferay CMS and Portal Meetup

Just helping to spread the word...

[Luis is] very pleased to see the amount of interest in our humble get-together. We have 26 members in our group of which 17 have said they're journeying to Ontario for our first Meetup.

Here's a quick update as to where we stand:

We have a venue and preliminary hotel and transportation information up on the site. We're working on possibly obtaining some group rates, so please refrain from booking a hotel, though we do suggest you purchase plane tickets to Ontario airport (ONT) as soon as possible as those never go down in price.

Transportation and Hotel Details may be found at:

The list of suggested discussion topics is also growing. Not all these topics are set in stone and may be reshuffled depending on attendee interest.

Agenda details may be found at:

This promises to be a fun and educational event, but we need YOUR help to ensure it succeeds. Please feel free to the meetup group and use its mailing list to connect with others, offer suggestions and feedback.

We also encourage you to spread the word about the meetup among your friends and colleagues.

If you have a blog or know someone who does, feel free to post about us to help spread the word.

The following link is probably the best one to share with others.

Alfresco + Liferay: West Coast Community Meetup

Join us on July 18th, 2007 in Ontario, California for our first CMS and Portal community event. Learn more at: end quotation
-- Luis Sala in Alfresco Fresh Talk

Perst Added to OSS Linkblog

And from this email, copied with permission, here's why:

Here's the connection --

You are correct that as an embedded database, Perst is not an Enterprise
database such as Oracle or DB2 and is not likely to serve as the main
repository for a data warehouse or business intelligence system.

However, Perst's high performance, its minimal RAM and CPU footprint, and
its pure Java construction make it an important component in systems that
optimize and manage business intelligence, data warehousing and other
enterprise applications. Its job is typically that of managing metadata or
providing a real-time cache, and the open source Perst plays this role in at
least two high-profile commercial software packages that are critical to
business intelligence and data warehousing systems' success:

* GigaSpaces embeds Perst to provide persistence for applications that
are deployed and optimized using the company's highly scalable,
self-managing distributed solution. Perst's all-Java architecture was a good
fit to provide maximum portability across diverse enterprise platforms, and
to take advantage of advanced Java capabilities. (see

* Wily Technology, a division of CA(NYSE: CA), uses Perst in its
industry enterprise application management solutions to store real-time
event data sent from remote agents, and to create a repository of
performance data, to enable analysis and optimization of enterprise
applications (including business intelligence and data warehousing). Wily
credits Perst with lending a 10-fold increase in data storage performance.

(And really, on under "Linkblog" in the
Databases section, you list BerkeleyDB and Firebird -- those aren't exactly
enterprise databases either - BerkeleyDB even bills itself as "The world's
most popular embeddable database engine").

Any chance of the open source Perst joining the open source DBMSs on that

Ted Kenney

end quotation
From: Ted A Kenney
Subject: RE: TeleInterActive Network: Perst open source database
Date: 2007 June 6 09:29:36 PDT
To: Joseph A. di Paolantonio

From OSBC2007 SF Is freedom the right approach to BI

Clarise and I met with Gabriele Ruffatti and Grazia Cazzin, from Engineering Ingegneria Informatica S.p.A. to discuss, and as Clarise wrote, get demos of SpagoBI [a full BI suite], and see the new Spagic EAI tool that was announced at OSBC2007, and the Spago framework [think Spring, only different].

Once he returned to Padova, Italia, Gabriele sent out some thoughts that came to him after participating in the open source business conference. He kindly gave me permission to publish them here.

"I attended Eben Moglen’s speach at the last Open Source Business Conference in San Francisco (May, 22nd). Hearing so many times: “stand up for freedom”, I thought: "I’m here now to promote free (in Europe we say libre as well) open source projects in different domains (Spago, java framework; SpagoBI, Business Intelligence; Spagic, SOA environment; Spago4Q, a Business Intelligence domain specific solutions) at an event mainly presenting commercial open source". While there, I’ve promoted new enhancements of SpagoBI, the Business Intelligence Free Platform, comparing it to other commercial open source choices such as Pentaho and Jasper and I’ve thought again: "what is the key differentiator? Stand up for freedom…"

"A participant said to me: “Guy: you’ve made a mistake. SpagoBI is not the Business Intelligence Free Platform; it’s the Free Business Intelligence Platform”. Was he right? I think no. This is the key: SpagoBI is a free platform, commercially supported, offering a new choice despite many other commercial open source products claiming more effectiveness to the market.

"What I mean by "commercial open source product" is: a solution claiming to be open source, claiming to have a community supporting it, but offering closed add-ons for enterprise adoption with a proprietary approach to the market (i.e.: acquisition of projects and IPs to strengthen its stack, dual licensing approach, aggressive marketing). Is it effective for the BI domain? Yes, probably it is.

"What I mean by a "free platform" is: free design, free collaboration, free assembling, free adoption. Not just because the license (SpagoBI adopts the GNU LGPL license), but because the efforts are in the software development improvement direction rather than in marketing proposition, in collaborations with different projects and solutions instead of acquisitions, in integrations with many free, open source, and also closed solutions to achieve the most effective solution for the user, maintaining a totally free code base core, instead of closed specific add-ons. Is it effective for the BI domain? I really don’t know, but the market, or a very new market knows the answer.

"Stand up for freedom. Is it “against” a wide commercial adoption? Is it “against” the growth of a strong business ecosystem? I know that Europe, Asia and South America are looking for free/libre software; Public Administrations are looking for free/libre software; new domain specific applications can be built thanks to free/libre software (Spago4Q is a just a first sample of it). At OSBC in USA, I’ve heard not only of commercial open source packages, but also of “building the right solution with an assemble mindset, opposite to a buy mindset”. It’s just a new choice: with an open mind, you can build open applications offering freedom to invent, to share your own knowledge, to assemble the right solution for people’s needs."
end quotation
-- Is freedom the right approach to the Business Intelligence domain? from Gabriele Ruffatti, Director - Architectures & Consulting, Engineering Ingegneria Informatica, S.p.A.

F/LOSS - Free/Libre… Open… "Free as in speech, not as in beer" is well known. But how open is open? This is a question that continues to be debated in terms of whether or not a company is truly open source, pureblood or mudblood. Andy Astor puts forth the rationale behind EnterpriseDB's strategy around openness [NB: EnterpriseDB is a company that is sometimes cited as a company using open source without being open source]. We tend to be fairly liberal in our acceptance of what is an open source company [including EnterpriseDB], while the market for enterprise open source defines itself and "commercial open source" has more denotation than connotation within that market, and I rather like the direction being taken in Commercial Open Source, cited above, by Carlo Daffara. Others may be more stringent in accepting a company as truly F/LOSS or not.

Most open source projects avoid locking their users into their product by following or providing open standards and open APIs, and/or using a plug-in, or framework, stack or platform architecture. Such an architecture allows the users and the projects community to more easily customize the product to suit their unique needs, and to more easily give-back to the project, without necessarily becoming an approved committer to the core code. We're currently following 47 open source projects that provide components that can be used to build solutions for data management and data analytics, data warehousing and business intelligence. We're following 5 open source BI suites, most of which use or can use the same core components. JasperReports for the reporting engine and Mondrian for the OLAP engine are common but not all-pervasive. All five suites provide a convenient stack of components that work together "out-of-the-box", but provide varying facility for including additional or replacement engines and components. We're still working out what is the real differentiator among all these tools, but Gabriele has a very good point: the more freedom, the more openness, the better.

Personally, I've been working in data analysis for nigh onto thirty years. The reason that we've been looking for open source solutions for decision support, data warehousing, data mining, BI, GIS, EAI, BPM, [insert latest hyped term here], etc, is that this area requires more customization and user involvement than any other enterprise application. The beginning of this millennium saw a trickle of open source projects addressing BI needs, and 2005 saw an explosion in the number of projects. Open source BI provides the flexibility that is always needed in satisfying the user needs for data analysis, visualization and sharing. The greater the freedom, the greater the openness, the greater the flexibility… the easier it is to satisfy the users. This is why data management, analytics and BI is going to be one the most successful markets for enterprise open source, bar none.

October 2019
Mon Tue Wed Thu Fri Sat Sun
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31      
 << <   > >>

At the beginning, The Open Source Solutions Blog was a companion to the Open Source Solutions for Business Intelligence Research Project, and book. But back in 2005, we couldn't find a publisher. As Apache Hadoop and its family of open source projects proliferated, and in many ways, took over the OSS data management and analytics world, our interests became more focused on streaming data management and analytics for IoT, the architecture for people, processes and technology required to bring value from the IoT through Sensor Analytics Ecosystems, and the maturity model organizations will need to follow to achieve SAEIoT success. OSS is very important in this world too, for DMA, API and community development.

37.652951177164 -122.490877706959


  XML Feeds