Nektra Advanced Computing is glad to announce that we are always working on Windows Live Mail API improvements from all your feedback. Now in our Windows Live Mail API Trial you are able to access the storage folders with read, write, delete, move, rename and clone capabilities. We are also able to append text and or HTML code inside each cloned message. Here is a screen shot of the Message Bodies in our WLMailApi C# Demo and below a video showing more capabilities of our product.

Contact us to ask any commercial question or use our forum for technical inquiries.

Nektra has released the new WLMailApi v1.0.1, which allows developers to create custom Windows Live Mail plug-ins.

Nektra Advanced Computing is glad to announce the release of the new WLMailApi version 1.0.1 which you can request a download by clicking here. This new version of WLMailApi left the Beta stage and is now an official RTM version.

Creating addons or plugins for Windows Live Mail Desktop® demands a deep knowledge of its internal behavior. Its Application Programming Interface (API) is undocumented and there is not a SDK available. Even the public interfaces IStoreNamespace and IStoreFolder supported in both Outlook Express® and Windows Mail® are not present in this new email client.

Windows 7® will not include Outlook Express or Windows Mail; its users will be forced to move to Windows Live Mail. Your product can be one of the first that plugs in!!!

Contact us to ask any commercial question or use our forum for technical inquiries.

This is the change log from previous version:

* Exiting from the system tray icon’s context menu crashed WLMail
* A stack overflow was produced when many database notifications were triggered
* There was a memory leak when handling database events
* C# DLL Demo solution made WLMail freeze after the evaluation expiration message when it was executed in Debug mode
* First events could get lost at startup before the first folder/message selection change event
* WLMailApi::GetCurrentMessageID() always returned -1 at startup until message selection was changed
* WLMailApi::GetCurrentMessageID() and NktWLMailApi::GetFirstSelectedMessageID() responses were not updated after current/selected message was deleted
* The WLMailApiEvents::OnDatabaseChange NKT_TR_INSERT_MESSAGE case was not triggered when messages were inserted in the Deleted Items folder
* Message.Delete(0) did not work when the message was in the Deleted Items folder
* WLMailApiEvents::OnCurrentMessageChanged event was not triggered if the current message was changed by selecting another message using “Ctrl + Click”
* When an unread message was selected in WLMail and was automatically marked as read, two WLMailApiEvents::OnDatabaseChange events (first NKT_TR_UNREAD_MESSAGE and then NKT_TR_READ_MESSAGE) were triggered instead of just one (NKT_TR_READ_MESSAGE)
* ToolbarButton::SetName and SetTooltip did not work if were called after the button was created
* Message::SaveAsFile() output file was corrupt if Message::GetHeader() was previously called
* Improved and corrected C++ and C# DLL Demos
* The solutions of the package now require Visual Studio 2008

The Nektra staff wishes you all a happy new year!

Best regards,

Nektra’s WLMailApi Team

We are glad to announce our first demo version of our new product Nektra Panorama. This toolkit allows developers to build applications in WM using controls built over the native API.

Nektra Panorama expands the platform with a fast-response user-experience and a very simple programming interface.

The grid view control is versatile, it can be filled with almost any thing. Here you can find a screenshot:

For a bigger example of the possibilities you can take a look our Nektra Contact Manager

Transitions

A high point of Nektra Panorama is the powerful transition interface. Using a single OS window for the entire application let developers create smooth transitions. Here you can see a ‘move’ transition to switch between menu window and Animation window in Nektra Panorama demo

The code to write this transition:

NktTransition* actionNew = new NktMoveToTransition(final, 600);

NktTransition* actionCurrent = new NktMoveToTransition(finalCurrent, 600);

wnd->MoveTo(origNew);

wnd->BringToFront(false);

actionNew->SetTarget(wnd);

actionCurrent->SetTarget(_current);

_scheduler.Add(actionNew);

_scheduler.Add(actionCurrent);

In this transition the origNew contains a rectangle out of the screen on the right. The rectangle finalCurrent is out of the screen on the left side. This code moves the control ‘_current’ out to the left and the control ‘wnd’ to enter the screen from the right side.

You can see a complete description of your product here

I was tired of some problems I experienced with the Thunderbird database and the lack of support of Windows Desktop Search.  So I decided to migrate from Thunderbird to Outlook.

I found several tutorials explaining how to do it using the free tool IMAPSize.

The problem with this procedure is that email conversion using this tool requires you to manually select each Thunderbird directory that contains the messages of each folder. It is fine for a few directories but a very long task if you have more than a hundred.

In addition to this issue, I found some export problems using the tool: it works with most of the messages but there are some problems with certain messages that are created with a wrong structure or with problems in the header (e.g.: empty subject and wrong received timestamp) when importing into Outlook Express / Windows Mail.

Looking for another method, I found a free tool Tbird2OE from PractiSoft. It is rudimentary but it does the first job: it exports all the messages to eml format keeping folder structure and no errors.

To import messages into Outlook Express / Windows Mail, the tutorial suggests to drag them in each exported folder. Again, this is simple for small amount of folders but it could take considerable amount of time if you have lots of folders.

For that reason I wrote a tool to import the exported messages using our product OEAPI to store messages in Outlook Express / Windows Mail.

Here is the step-by-step tutorial:

1. Compact messages (recommended but optional)
2. Make sure you have lots of disk space free to store the messages (from Chuck).
3. Use Tbird2OE to export messages from your Thunderbird profile:
   • Top edit box your mail root in the top edit box, usually something like this: C:\Documents and Settings\UserName\Application Data\Thunderbird\Profiles\awfiwoeu.default\Mail\Local Folders
   • Bottom edit box: any path where the emails will be stored in eml format keeping folder structure
4. Install OEImportEml:
   • Set the same directory that you specified in Tbird2OE as export path (the bottom edit box)
   • Set Thunderbird ‘Sent’ folder if your installation is not English (the folder specified here is forwarded to ‘Sent Items’ in Outlook Express / Windows Mail)
5. If you want to use Outlook, run Outlook and select to import messages from Outlook Express / Windows Mail

Download Deviare
Download Sourcecode
Download PDF

Contents

• Introduction
• Research
  • Direct
    Sound Capturing
  • Monitoring Skype Conversations
• Implementation
  • Deviare
    Python Wrappers
  • Wave Tools
  • COM Type Libraries
  • Virtual Table Finder
  • Hooking Direct Sound
• Running Sound Capture
  • Easy Steps
  • Registration
• What’s next
  • Optimizations
  • Wave API Hooking
  • Hook DirectSoundCapture And Listen To Full Conversations
  • Inspect More COM Interfaces

Introduction

Today we are going to see how easy it can be to capture audio with Deviare. From players like Windows Media Player, instant messaging applications like Skype & Windows Live Messenger, to any application using DirectSound. The wave output will get captured by us.

Deviare is indeed a powerful framework. Built to resolve most complex tasks in the simplest way. With a few lines of python, all our hooking is done and running. Today performance is extremely important, yet Deviare proves itself as the best. It allows you to also take advantage of the full power of Python

Research

Direct Sound Capturing

I must be honest, I’ve never used the DirectX API in my life, so I was a bit uncertain of how difficult this could be.
I started by looking at MSDN documentation onIDirectSound and IDirectSoundBuffer.
The first goal was to find a safe place to read its sound buffers. I found out that IDirectSoundBuffer::Unlock could serve my intentions well. At this point, the user is telling DirectSound that he has finished writing his wave output and the locks may be released. So, if we step in between, we can safely read the buffer. The user is no longer writing to it, and DirectSound has not yet taken control of it.

I tested it on many applications and it turned to be the right choice.
It works perfectly for WMP, Windows Live Messenger, and many others.
No problems showed up until I stepped with Skype…

Monitoring Skype Conversations

This might be the way many applications handle their sound output, but it was the first application I have seen and I named the case after it.
Later, I found a few articles describing it in detail.

So, to my surprise, I was not seeing any data being written to the sound buffers after the unlock was called. How the hell is it writing its wave, and how am I supposed to read it?!. It kept me thinking for a while, until I noticed an interesting and constant call to IDirectSoundBuffer::GetCurrentPosition. Then I realized that this writing method depends on constant reading of the play and write buffer pointers. That’s because DirectSound, as most stream based implementations, works with a Circular buffer. Capturing its wave output requires that we keep track of changes in the write pointer. Once we know it has moved forward in the buffer, we can read its steps. Since here we don’t know how much the user has actually written to it, we must know the full size and location of the buffer. Unless we want to read garbage, but I’m sure that’s not the case ;).

Implementation

Deviare Python wrappers

Before we get our hands dirty, let me introduce you to something new in Deviare: Python Wrappers. As you already know Deviare is exposed through a series of COM interfaces. To save ourselves from the work of writing a whole new set of bindings, we used the well known project PyWin32. It’s very friendly to be used directly, as you may see in py_deviare_objects.py, just not enough to me. So I built these wrappers on top of it and made them as transparent as possible.
You’ll find the use of the interface very similar to the way it’s done in our C# examples and in compliance with the python way of life, of course.

Wave Tools

I wrote these tools to help me write down the captured wave data. This may be obvious to people working in audio projects, but for me I cannot believe there is no native support in Windows to read-write Wav files! Yes, there is native support to write RICH content but come on!

Luckily for me, I found a small sample C++ class inside the DirectX SDK. This was good enough for me to write my own in Python. As you may see, my WaveFile class only supports the write operation. Though, adding a read member should be easy enough for you :). I have also added a lock to it, to ensure our data does not get corrupted by multiple thread operations. You may use it safely.

The structures used were defined exactly as found in DirectSound and WinMM headers. Some of them are used by DirectX to specify the format of the wave content.

COM Type Libraries

By default, DirectX installations do not register their library types. Since we need that information, so Deviare can hook them, I created my own definitions with the interfaces we are interested in. To prevent any collision with previous installations, I used a different GUID. There is a python script that takes care of its registration and it’s automatically ran on demand by our example. Again, definitions are exactly as found on DirectX SDK.

Virtual Table Finder

To obtain the virtual tables for the interfaces, we basically have two options. Either we wait for its instantiation by the target process, or we find them ourselves on our own. Our first option is known to work for sure, yet we delay our installations until these events rise. This may also place us in a race and we may not capture all the output. The second one allows us to hook our targets immediately. Yet, in this case, we depend on the library (dsound.dll) being loaded in the same address space of our target. I have placed the two options in our example. If the current one is not working for you, uncomment the other at py_deviare_directsound.py.

Hooking Direct Sound

The first thing we need, is to know every time a sound buffer is created.
For that we are going to intercept calls to IDirectSound::CreateSoundBuffer.
If the calls succeed, we look-up the table location inside the returned instance.

From there we are going to hook four members of IDirectSoundBuffer:Initialize, SetFormat, Unlock and GetCurrentPosition.
The first two, are used to obtain the wave format that the user is writing to the buffer. We also need to watch details from the call that creates the sound buffer, in case it is specified there.

The Unlock member, as our research told us, is used in most applications to notify DirectX that the buffer is written and ready to be played. So we read the buffer pointers and size, to use Deviare’s memory interface to copy all content. We need to be careful, and see if the call actually succeeded. Only then can we save the wave data, else we must discard our buffers.

With applications that keep track of the play and write cursors, we are going to monitor their calls to GetCurrentPosition.
As explained earlier, with this method, I need to know the full size of the buffer and its location. So I save it from the first call to Unlock. Then I virtually divided my buffer in N segments, and filled it with the wave data as the write cursor moves forward in the buffer. Once my buffer contains enough data, I write it down to the wave file. To prevent false positives, in the creation of sound buffers, I delay the creation of my file until I have real data stored.

In case we are monitoring the creation of IDirectSound in the target process, we also need to hook DirectSoundCreate and DirectSoundCreate8 from dsound.dll.
There we can obtain the virtual table for IDirectSound, and follow our quest.

Running Sound Capture

Easy Steps

Execute the run_me.py located among the deployed files, and you’ll be prompted with a window to type the complete name of the process you want to start monitoring. For example: Skype.exe. Once the program starts capturing, the wave files will be written in the same folder.

Once you are done, click OK on the dialog box to stop recording. Now you can open the .wav files generated, and listen the capture. Do not open them before closing the example as the data may not be readable by then.

Registration

The first execution of our example, will automatically register its interfaces and data types. It will also generate a file labeled .deviare_types_registered to prevent registration on the following executions. You can safely remove the file at any time you want the registration to be run again.

What’s Next

Optimizations

At any point of our handling, performance is essential. Any delay is highly punished by the sound output. So we must be careful about any operation we do inside the function call. This example tries to cache enough data before doing a write operation to disk. In case you need to improve its performance, you should read the data and release the call as soon as possible. Then in a different thread, or in a non punitive call, flush our data to the wave file.

Wave API hooking

This example could be very easily adapted to capture wave data from applications using WinMM API. Most browsers, Flash, and Google Talk use it to throw their sound output.

Hook DirectSoundCapture And Listen To Full Conversations

You should have noticed, when capturing from Skype, that your own voice is not heard. That’s because the application is not echoing its capture from the microphone. To get that too, it is necessary to hook IDirectSoundCaptureBuffer and proceed the same way to read its buffer.

Inspect More COM Interfaces

If you want to discover a lot more about the internals of DirectSound, then Deviare will be very valuable for you. Inspecting COM object is very easy indeed. Simply define one of its interfaces (if its not already registered in the system) and hook them the simpler way.

If you are wondering what other interfaces may be useful for you, try our Deviare COM Console to discover them. It comes with source code, and you are free to adapt it to your needs!

And That’s All Folks, hope you find it useful, enjoy!

Download messagespy_demo.zip - 250 KB

Download messagespy_src.zip - 249 KB

Deviare Message Spy

Contents

• Introduction
• So, what’s the good news?
• Deviare Message Spy
  • Finding a Window: The Spy++ Style Window Finder
  • Hooking
  • The XML
  • The Cast
• Using Deviare Message Spy
• Known Issues
• Resources

Introduction

This article presents you with a different perspective of how to inspect window messages, to see how applications are communicating and managing their controls. We are not going to explain what window messages are or what they are used for in this article, so we suggest that you read these excellent articles to understand them: Handling Window Messages (Part 1, Part 2, Part 3). In this article we are going to monitor the Message API from the inside by hooking the target process.

So, what’s the good news?

As a first step when developing, to inspect windows, we open the Spy++ application and start the tedious work of following messages as they are printed in their hundreds. This is helpful most of the time, as we usually want to know what our windows are seeing and receiving. Yet, what happens when we want to know exactly how an application is communicating with its controls (what calls it makes to the message API) or want to see if our messages are getting filtered by someone else? As you may know, Spy++ installs 3 global hooks to receive every Send, Post and Call to a window message handler. The information provided by these methods is not enough to know what messages are coming from our application or if any of them have been filtered by a hook installed earlier in the call chain.

Do not panic, Deviare comes to rescue. What we are going to do is intercept all the Message APIs from the process that the window belongs to and monitor its calls. From there, we can be sure of what messages are being sent from the application to its controls and if any of them are missing from the ones that Spy++ is reporting, then we will know if someone else is watching us…

What happens with the messages not known by Spy++? How are we going to see them? Look what happens with many of the messages used by the standard ListView in windows. Spy++ does not know anything about them if the window is subclassed (for example ATL:SysListView32), and cannot trace its content. Try following LVM_GETNEXTITEM in Outlook Express and you will only see unknown 0×100C messages. The same goes for custom user messages that you may know and want to follow. We need an application that can be customized to our needs!

Deviare Message Spy

To probe our theory, we have built this message spying application. We have added to it a way to lookup windows handlers, hook the process owning it, and correctly report the messages and structures.

Finding a Window: The Spy++ Style Window Finder

To pick the target window and the process we wanted an interface like the one used in Process Explorer and Spy++. Thanks to Mark Belles this was an easy task. He has a great article on how to implement a nice Window Finder, in Code Project.

Hooking

In order to install a hook, first we need to identify our target process. After obtaining a window handle from our Window Finder, we can use GetWindowThreadProcessId to identify which process owns the window. From there we use the .Net API to access it and tell Deviare which process we wish to hook.

Win32.GetWindowThreadProcessId(hWnd, out _processId);
_txtProc.Text = Process.GetProcessById(_processId).MainModule.ModuleName

For our monitoring we have divided the API in 2 sets: the Dispatch group, and the Sent and Post group. Monitoring messages that arrive to the first group will provide us with a very similar view of what Spy++ sees. This is because these messages arrived to the application and have not been filtered by any hook. With our second group, we will identify direct and asynchronous calls to the Message API.

Let’s see how we install the hook for one of these functions:

procs = _mgr.get_Processes(0);
procs = _mgr.get _Processes(0)
proc = procs.get_Item(_processId)
IPEModuleInfo mod = proc.Modules.get_ModuleByName("user32.dll");
IExportedFunction fnc = mod.Functions.get_ItemByName("PostMessageW");
_hook = _mgr.CreateHook(fnc);
_hook.Attach(proc);
_hook.OnFunctionCalled += new Deviare.DHookEvents_OnFunctionCalledEventHandler(_hookPst_OnFunctionCalled);
_hook.Properties = (int)DeviareCommonLib.NktHookFlags._call_before;
_hook.Hook();

As you see, we easily pick our target process by Id and select its Module and Function by name. The module name is not important, as it is always going to be “user32.dll”. If you have doubts, you can use Spy Studio to watch the process modules and exported functions.

Once the hook gets installed, we will receive notifications on our handler. From there we parse the function parameters transparently with the interface provided. (These parameters are actually in the target process, and Deviare copies them to our process on our demand and handles all the communication).

int returnVal = callInfo.ReturnValue;
IParams pms = callInfo.Params;
IEnumParams enm = pms.Enumerator;
IParam pm = enm.First;
IParam recvMsgHndl = pms.get_Item(0);
IParam recvMsgParam = pms.get_Item(1);
IParam recvWParam = pms.get_Item(2);
IParam recvLParam = pms.get_Item(3);

After reading all the data we require from the call, we will use our generated Xml to identify the message and properly cast it to its structure and show it properly.

The XML

The XML document in this application was created specifically to link together the message names, values and parameters. As messages like WM_LBUTTONDOWN are predefined as 0×201 we can place this in a XML file containing information on the parameters WPARAM and LPARAM.

<message value="0x201">
<name>WM_LBUTTONDOWN</name>
<return value="">
<returninfo></returninfo>
<returnmisc></returnmisc>
</return>
<wparam value="">
<wname>wParam</wname>
<wmisc>wParam Indicates whether various virtual keys are down. This parameter can be one or more of the following values.
MK_CONTROL
The CTRL key is down.
MK_LBUTTON
The left mouse button is down.
MK_MBUTTON
The middle mouse button is down.
MK_RBUTTON
The right mouse button is down.
MK_SHIFT
The SHIFT key is down.
MK_XBUTTON1
Windows 2000/XP: The first X button is down.
MK_XBUTTON2
Windows 2000/XP: The second X button is down.</wmisc>
</wparam>
<lparam value="">
<lname>lParam</lname>
<lmisc>lParam
The low-order word specifies the x-coordinate of the cursor. The coordinate is relative to the upper-left corner of the client area.
The high-order word specifies the y-coordinate of the cursor. The coordinate is relative to the upper-left corner of the client area.&amp;amp;amp;amp;lt;/lmisc&amp;amp;amp;amp;gt;
</lparam>
<misc></misc>

We could not find any database with this information, so we generated an XML document with the messages that we were interested in knowing about. As you can see, it is easy to simply add any message you want. In the process of building this XML, we used a very nice tool called ApiViewer from ActiveVB.de. Just search for the message names you want and you can evaluate the message values from the names.

The Cast

Now that we can identify the structures used on messages, we need to tell Deviare. Basically we are telling it to interpret our parameter, not as a simple LPARARM or WPARAM type, but as the complex structure we know is there. This is the case for messages like WM_DRAWITEM. So, to read it’s structure contained within the LPARAM we need to cast it as follows:

IParam pm = pms.get_Item(2); //LPARAM
pm = pm.CastTo(“LPDRAWITEMSTRUCT”); //Now our IParam is read as a pointer to DRAWITEMSTRUCT
pm = pm.Evaluated; //Resolve the pointer indirection
//Ready to use IParam as the structure sent by the OS.

It is possible to do this with all of the structures you can find defined in the windows headers. So, you should be able to cast and read any of them that are used in within these messages.

Using Deviare Message Spy

Deviare Message Spy in Action

Above we have our Deviare Message Spy in action. We selected the contacts list window from Outlook Express (at the bottom left) to spy on. You can see all the message values that were sent via Post and Send Message APIs. LVM_HITTEST has been expanded to show the full values received. As LPARAM is a pointer to the LVHITTESTINFO structure we can find all relevant information contained within.

Hope you enjoyed this article, and found it useful. Let us know what you think!

Known Issues

Many messages have the same Hex Address, such as TB_GETITEMRECT and TTM_UPDATE. Both of these messages have the value of 0×41d but are very different messages.

The TTM_UPDATE Message Forces the current tool to be redrawn. It does not use the wParam and lParam where as TB_GETITEMRECT Message Retrieves the bounding rectangle of a button in a toolbar.

TB is a Toolbar message and TTM is a Tooltip message. As our Spy++ style window finder already finds the window class, such as SysListView32 and ToolbarWindow32, It would be easy to use the class name to tell the program with Xml message is the correct one.

Resources

• Deviare API Hooking
• Message Lookup: ApiViewer
• Spy++ from the Microsoft Windows SDK

We are comparing our hooking engine Deviare with some of the other products available, so that you can get an idea of what each engine can provide.
Here is a comparison of Deviare against Easy Hook.

Functionality	Deviare	EasyHook
Database with Functions and Data Types	Yes	No
Intercept multiple functions with a single handler	Yes	No
Selective Handler for each Function	Yes	Yes
Relocation of Relative ASM Instructions (RIP)	Yes	No
Save & access Call Function Context	Yes	No
Access registers & flags	Yes	No
Access return address	Yes	Yes
Get/Set win32 last error	Yes	Yes
Monitor COM Objects creations	Yes	Partially
Hook COM Objects	Yes	No
Provides COM Interfaces accessible from any language	Yes	No
Enumerate process’ modules	Yes	Yes
Enumerate module’s exported functions	Yes	No
Get module path and info	Yes	Yes
Automated call of original function	Yes	No
Thread Deadlock Block	Yes	Yes
64 bits support	No	Yes
Thread Safe Hook Install	Yes	No
Native Support	Yes	Partially
Inter-Process Communication	Yes	Partially
Custom Library Injection	Yes	Yes
Stealth Support	No	Yes
Kernel Mode Hook	No	Yes
Driver Installation	No	Yes
Relocation of Instruction Pointer	Yes	No
Requires .Net Framework	No	Yes
Use System Runtimes (CRT)	Yes	No
Hook Terminal Sessions	No	Yes
Full unload before target termination.	Yes	No
Execute As Service	No	Yes
Prevent execution inside OS Loader	No	Yes
Thread selection filter	No	Yes
StackTrace	Yes	Yes
User-mode Wide Hook	Yes	No

We all remember when Ole Automation came out. We were all impressed how simple it was to implement a few COM Interfaces, place a toolbar and interact with the office package. Soon the competition began to show who could create the best and most creative Add-on. How many times did you wonder how that other plug-ins “did that”? What if now you can even know how Outlook, or any Office application operates? Well, my friend, take a closer a look…

This Deviare example is implemented as an Outlook Add-on. We have used C# .Net, but you can use any language that supports Component Object Model.

We are using 2 threads to avoid freezing the application. The first one is the standard thread where Outlook report its events to us. The second is our worker thread where we create an output window to print our messages and a Deviare Event Proxy to process functions’ calls.

From the events Outlook provides us to work with we are only interested in OnStartupComplete. Here we know that Outlook is done with all its initialization and we can start hooking its interfaces. As a regular plug-in we ask for the Outlook Application, Active Explorer, CommandBars and create a CommandBarButton. We are going to intercept all of them and see how their members are used.

Notice that to obtain the interface we don’t use the class implementation, but the underlying interface definition. That’s why, when calling HookInterface, we send the Type of Outlook._Application and not Outlook.Application. The second one, is the .Net wrapper, and the first one is the Ole Interface.

To intercept these objects, Deviare needs some information. The necessary elements are the COM Object Interface (that would be its virtual table), which members we are interested in (specified by index), and the name of the Interface. Identifying the interface by name, will let Deviare find all the information it needs during the call, so you can handle its parameters the same way we did with any function hook. To gather all this the .Net Framework provides us with marshaling facilities (System.Runtime.InteropServices.Marshal), this makes our lives pretty easy ;).

And that’s all. We print our calls, and see our results:

Cheers, and happy coding!

Everyone likes something for free, but open source is free as in freedom not as in beer.

Lots of companies use open source software and many put a lot back into the open source world but just how much do they put back?

It is impossible to find a definitive answer to this so we are just pointing out some ethical issues, when a “leecher” takes the work of others without giving back its detrimental to the open source movement.

In 1993 Apple’s Operating System was seriously in need of an upgrade. Their internal development of a new OS was not going well, so they looked externally for an OS.  BeOS and NeXTSTEP were the main candidates,  BeOS was a completely new OS developed from scratch and NextSTEP was an OS built on top of BSD.  They went with NextStep and acquired the company behind it NeXT for $429 million.  BeOS was later bought by Palm, inc after which they discontinued BeOS.  BeOS users without a viable upgrade path and BeOS developers with programs stranded on an unsupported platform then decided to develop Haiku, an Open Sourced operating system.

NeXTSTEP then went on to become MacOS X.  Apple released the Open Sourced Darwin, The core components of MacOS X, in 2000 under the Apple Public Source License.  This release spawned OpenDarwin, a project designed to create a stand alone Darwin operating system.  This failed with the developers stating OpenDarwin had “become a mere hosting facility for Mac OS X related projects.  Availability of sources, interaction with Apple representatives, difficulty building and tracking sources, and a lack of interest from the community have all contributed to this.” There is a new project called PureDarwin which is currently trying to complete a release based on Darwin 9.

Apple used to distributed a binary release of Darwin themselves but stopped in 2005.  Currently they only release the source code of Darwin, Although within this they include proprietary drivers of their AirPort wireless cards.  They also exclude Carbon, Cocoa, Quartz Compositor and the Aqua user interface.  This prevents users from running MacOS X software.  There were good reasons for Apple to go via the closed source route for these but when taking so much from the Open Sourced Community surely it would be nice to give more back? Although some people say Apple have saved BSD.

Microsoft is a company well known for their closed source software.  Originally network protocols were an add on to their operating systems and it wasn’t until windows 95 that the TCP/IP Stack became part of the operating system.

Originally they tried their own protocol Netbeui and then reverse engineered Novell’s IPX protocol but finally they adopted the BSD’d TCP/IP (you can see the BSD license within their source here.) There is nothing ilegal in Microsoft using the BSD code for their implementation of TCP/IP but because the code was under the BSD license any changes they made were not released for the benefit of the rest of us.  If the code had been under the GPL license would they have used it?  Who knows but it would have forced them to release any updates they made.  Maybe they would have developed an alternative protocol and the internet would not be as it is today!

Flock is an interesting case of a company using open source for the basis of their product.  Flock itself is an open sourced project where

“Yes, Flock will be open source.  We may incorporate some proprietary technologies into our browser and releases some features under a commercial license, but all of our initial code, and the vast majority of our code going forward, will be open source.”

We know the Flock developers want to keep Flock compatible with Firefox and it may be based on Firefox, but they are two separate projects.  Code will increasingly change and although extensions currently work on both browsers as the code develops we’ll see extensions working with either Firefox or Flock.  What will happen if Firefox decide to implement some of Flock’s ideas differently?  Will Flock go to their source and follow the way Firefox have implemented the idea?

Would it have been possible for the Flock developers to have just released an extension that just adds the tools that Flock adds?  With the creation of a new browser they have potentially forked the developers of extensions for Firefox.  Flock has investors to keep happy,  is it possible for them to do that whilst keeping extension compatible with Firefox?

What about Google’s File system? This is a customized file system that writes LARGE chunks of data and sits above a standard Linux file system.  What changes have Google made to the “standard” Linux file system? Are they going to give the open source community their code? As the software is running on a server and not being distributed they don’t need to share the code but should they?

We all know Google also uses a stripped down optimized version of the Linux Kernel for high performance without which they couldn’t possibly exist.  What changes have they made that they’re not sharing with the rest of us? The hiring of Andrew Morton does help though.

For example there are a number of companies that use dmoz.org data the most well known being Google Directories which combines the dmoz data with its own pagerank. Does this make dmoz irrelevant? How are they contributing to the dmoz project?

When money becomes involved it isn’t uncommon for a project to move from open source to closed source.  Activecollab was one such project,  originally released under an open source license, it has since moved to a closed source commercial project.
“When it was first released, activeCollab came with an open source license and that was what attracted me to the project. I thought it held promise of being a very powerful and useful project management application if developed by an active community of users.
But the developer has decided to stop open source development on the project. Development will now be closed source, at least on the core features. The next release, version 1.0 due out next week, will also not have a free version. Your only options for activeCollab 1.0 are SmallBiz ($199) and Corporate ($399).”

Although a fork has been started the move to closed source will have alienated a lot of developers and users.

It is interesting to watch Mozilla to see how their creation of a “for profit” corporation which supports the popular Firefox Web browser and Thunderbird E-mail client as well as developing custom software based on open sourced products.  This happened in 2005 now 3 years later has much changed?

“the Mozilla Corporation is not a typical commercial entity and will only pursue revenue-generating activities that are consistent with offering end-users with the best experience possible.”

Mozilla did ask the public for money in a 2 page advert on December the 16th 2004.  This was before the creation of their commercial wing but with reports of as much as $72 million being poured into Mozilla Corporate from Google for carrying Google advertisements why are they still asking for donations?

Should the people who donated before the corporate side was formed be considered shareholders of Mozilla corporate? What about new donators?  Why should you donate to an organization that has a very wealthy corporate company behind it?

Will all the contributors to Firefox and Thunderbird be given shares or money for their work that the Mozilla corporation are profiting off?

We can see that Firefox and Thunderbird are both still available for free from the Mozilla website, Both currently supported by Google’s Ad money and donations, but for how long?  With Firefox being the cashcow for Mozilla Corporate will Thunderbird fall by the wayside?  There are rumors that it will find itself under a different company with Mozilla Corporate becoming Firefox Corporate.

One companies solution is the advent of dual licenses.  They release their software as both open source under the GPL and as a commercial product.  This allows them to employ staff full time to work on the product, It also allows companies that wish to use the software and modify it but not share their code with others that opportunity.

In the Games Market id Software also license their older game engines under the GPL as well as offering the opportunity to use their engines under a commercial license. Other companies such as 3d Realms and Parallax Software have released the source code for some of their games but without a commercial option for their engines.  The dual license here obviously benefits companies wishing to add to the source code to the detriment of being forced to release possible upgrades to the original open source engines.

Another place where Open Source software has taken off is in Web services. Many companies are taking advantage of Open Source software on the Web but they’re not obligated to publish their code if they make any changes because they are not redistributing the software in a package as either a download or a physical medium.  The web is a new distribution medium for them.

A License has been developed to apply to the software that is in this loophole, The Affero GPL.  This licenses software that is ran on a server.  One such project is in the UK where petitions to the Prime Minister is ran on Open Sourced AGPL v3 software.

What is the fair value of return to the open source if you earn a lot of money?

“IBM says to a customer, ‘Do you want proprietary or open software?’ Then if they want open source they say ‘OK, you want IBM open source.’  It is always IBM or Sun or HP open source,”

“Companies are using the potential of communities as subcontractors — the open source community today is a subcontractor of American multinational” said Jesús Villasante, head of software technologies at the European Commission

in reply James Baty, a vice-president at Sun, said that companies such as his have a responsibility to contribute to the open source community.  Sun itself contributes to a number of open source projects, including the open source productivity application OpenOffice.org.

“There are companies that are takers from the open source community, other companies are taking the attitude that they have to contribute, Open source should be seen as an opportunity, not as something to capture and abuse.”

We know major corporations have made valuable contributions to open source software, as well as persuading businesses and IT professionals that it is a credible alternative to proprietary options, but do they take more than they provide?

We here at Nektra Believe Open Source will always have an important place in the world and provide CookiePie under GPL, NKT WAB under LGPL and Trappola under LGPL, but it is essential that the Open Source community is not taken advantage off and valuable contributions are put back into the community.