Nektra Advanced Computing Blog

This week Mozilla launched Firefox 3 officially. It was covered in a couple of sites like:

• It’s Firefox 3 world record download day (techcrunch)
• A Few Firefox 3 Followups (slashdot)

We have released a new version of Cookiepie firefox extension available for download, and we have a number of features and a long wish list from users that we hope can be introduced in next months. Also we are designing some T-Shirts for our Cookiepie fans! More updates will be notified in the Cookiepie mailing list

Using our API hooker SpyStudio I wrote a script to intercept http requests done using wininet.dll API coming from a specific module of a process. The script keeps request information (server and url) to display in next calls and let filter requests to a specific server. Its name is httpReport.py and can found in SpyStudio v1.0.1 distribution.

httpReport navigates the stack in each call to wininet.dll functions to see what module called the hooked function, filtering all modules except the specified. This feature and server name filtering, allow a fine interception.

To use the script keep only one instance of iexplore.exe (the script will only hook the first instance if there are more than one) and type these lines in SpyStudio python console:

import httpReport
httpReport.startIe(’toolbarqueries’, [’googletoolbar2.dll’])

The script will display queries done to a server that contains the string ‘toolbarqueries’ coming from module ‘googletoolbar2.dll’.

For example, if TechCrunch page is inserted in the address bar we get a wininet.dll!InternetConnectA call to ‘toolbarqueries.google.co.uk’ server and then a GET request to this url:

/search?client=navclient-auto&googleip=O;64.233.169.147;266&iqrn=ZjbD&orig=0PnmJ&ie=UTF-8&oe=UTF-8&features=Rank:&q=info:http%3a%2f%2fwww%2etechcrunch%2ecom%2f&ch=751153802320

There are some parameters that need more research to be understood but there are some others we can tell something:

googleip: indicates Google server used for the query

ie: iexplore encoding?

oe: maybe Outlook Express encoding?, only a bad guess

features: what we are asking to the server (here ‘Rank’)

q: encoded url (http%3a%2f%2fwww%2etechcrunch%2ecom%2f = http://www.techcrunch.com/)

ch: it looks as a function to the url to prevent other client to do the same requests

Then, wininet.dll!InternetReadFile return the http response:

‘Rank_1:1:8\n’

that indicates that the page visiting has PageRank 8.

This process is repeated for every page you visit so Google can collect all the pages browsed by all the users using GoogleToolbar. That’s why it may be considered as a spyware.

Seems that the Google guys are getting softy. The last two questions of the Google Treasure Hunt 2008 were easily solved.

The Question #1 is about paths. We have a robot that can move down or right, in a n x m grid. So how many possible paths exists, from the top left to the top right?

It gets solved just searching in Google for “grid path right down” from there you will get the equation that you must run on any language that has Big Integer implementations, since involves the calculations factorial.

Example of our solution for the first puzzle in Java:

BigInteger dividend = factorial( (rows-1)+(columns-1) );
BigInteger divisor = factorial(rows-1).multiply(factorial(columns -1));
System.out.println(dividend.divide(divisor));

The Question #2 seems to be even easier. It involves to transverse a directory tree, filtering the files that verifies 2 conditions based on the path string and the extension string (like .txt or .xml). Then reading some specific line. All files are text files this simplifies then things even more. Nothing hard to any programmer.

Snippet of our solution for the second puzzle in PHP:

// Setting where's the Google Treasure Hunt Directory
$dirbase = 'GoogleTreasureHunt08_11336769377172459175';

// Creating and loading the directory Tree
$tree = new Mytree($dirbase);
$tree->load();

// Getting the leaf Files
$leafs = $tree->get_leafs();

// Filtering to files that satisfies the conditions
$cond1 = array_filter ($leafs, filter_bycond1);
$cond2 = array_filter ($leafs, filter_bycond2);

// Doing the sums at the right line number
$sum1 = array_reduce($cond1, create_function('$v, $node',
             '$v = ($v == null) ? 0 : $v;'.
             '$v += (int)read_line($node->data, 5);'.
            'return $v;'));
$sum2 = array_reduce($cond2, create_function('$v, $node',
            '$v = ($v == null) ? 0 : $v;'.
            '$v += (int)read_line($node->data, 5);'.
            'return $v;'));

echo $sum1, '<br>';
echo $sum2, '<br>';

// Obtaining result
echo $sum1 * $sum2;

So as you see, there’s no complication at all. I would expect some challenge when Google uses the “Puzzle” word. Maybe they aren’t what they were? I don’t know, but I will be expecting some real challenge to solve :).

Robot solution:  GTH Q1 Java solution

File transeversing solutino: GTH Q2 PHP solution

Back in 2001 we wrote an arcane simple perl script just for fun. The script gave you the freedom to export IMDB movies from your ranking to your web page. Now in 2008 and with business goals in mind, we need to export/interact with information from sites like LinkedIn and add it to our handcrafted mashup pipeline or our CRM, but it’s not possible without being blocked, see LinkedIn Scraper for more information.

Data portability is a must (sometimes an API is enough) but the "official" data portability innitiative came very late:

> whois dataportability.org

…

Domain ID:D149678191-LROR
Domain Name:DATAPORTABILITY.ORG
Created On:07-Nov-2007 04:21:39 UTC
Last Updated On:07-Jan-2008 03:49:01 UTC
Expiration Date:07-Nov-2008 04:21:39 UTC

…

On the other side, sites like LinkedIn increase their business risks if they share too much data and become easily replaceable. So the question is how to deal with this conflict of interest? An obvious business model  is what SalesForce does with AppExchange and Facebook does with Facebook applications: Run your applications within their [eco]system.

So now in 2008 LinkedIn has been lazy to balance between being open or not (even putting a public price to their data sharing policies), but the data portability "lobby" woke up too late, or Late 2.0.

It is hard to find on the internet a detailed and complete solution for modifying the contextual menu due to several reasons.

One of these reasons is that many of the implementations found use the System.Windows.Forms.ContextMenu; you can see one of them here:

Component-Based Development with Visual C#

In these kinds of examples the system menu is not invoked from the ShowContextMenu, instead a user customized menu is. This menu does not allow modifying it as we need.

Another reason is due to the programming language. In the MSDN website a C++ implementation of the ShowContextMenu can be found:

WebBrowser Customization (Part 2)

The problem is that when we want to implement it in C# difficulties such as not being able to call system functions, use the same data types, and many others arise.

Maybe the biggest difficulty can be found when trying to marshall the CComVariant class. A huge variety of solutions can be found on the internet, but they usually do not work (at least in the case mentioned above). Here are some examples of them:

VB Variant Equivalent in C#

Object To Variant

What is the equivalent of Variant data type in C#.NET?

Using the int[] type with size 3 or bigger is one of the ways of solving this.

   1:  int[] variantVar = new int[3];

The VARIANT type can be seen in this MSDN webpage:

VARIANT and VARIANTARG

Once we solved this problem, we can use the IOleCommandTarget function Exec:

   1:  [PreserveSig]

   2:  int Exec(

   3:      ref Guid pguidCmdGroup,

   4:      int nCmdID,

   5:      int nCmdExecOpt,

   6:      // we need to have this an array because callers 

   7:      // need to be able to specify NULL or VT_NULL

   8:      [In, MarshalAs(UnmanagedType.LPArray)] int[] pvaIn,

   9:      [Out, MarshalAs(UnmanagedType.LPArray)] int[] pvaOut

  10:      );

When calling Exec for the first time, we get the handle for the language submenu. We obtain it in variantVar variable:

   1:  int[] nullVariantVar = null;

   2:  int[] variantVar = new int[3];

   3:   

   4:  spCT.Exec(

   5:              ref CGID_ShellDocView, 

   6:              SHDVID_GETMIMECSETMENU, 

   7:              0, 

   8:              nullVariantVar, 

   9:              variantVar

  10:              );

Now we must parse variantVar in order to get the result (the handle for the language submenu). The first value that we get is a VARTYPE type, which indicates the kind of variable that we will find next. Then there is a reserved spot of three WORD long, followed by the value we are looking for. So the handle for the submenu is on the second place of the array:

   1:  IntPtr handleSubMenu = new IntPtr(variantVar[2]);

We can replace passing the CComVariant argument to the function by creating a new variable shown in the code below and then call again Exec:

   1:  variantVarIn[0] = VT_INT_PTR;

   2:  // Remember that variantVarIn[1] is reserved

   3:  variantVarIn[2] = handleMenu.ToInt32();

   4:   

   5:  variantVarOut[0] = VT_I4;

   6:  // Remember that variantVarOut[1] is reserved

   7:  variantVarOut[2] = dwID;

   8:   

   9:  // Insert Shortcut Menu Extensions from registry.

  10:  spCT.Exec(

  11:              ref CGID_ShellDocView, 

  12:              SHDVID_ADDMENUEXTENSIONS, 

  13:              0, 

  14:              variantVarIn, 

  15:              variantVarOut

  16:              );

We obtain the complete context menu as a result of the instructions shown above. This menu can be modified as much as we desire. Using this, you can add or remove menu items and also their functionality. For example you can call methods implemented in your project from the desired menu item.

Now you can build a customized browser using C# !

Introduction to SpyStudio:

SpyStudio is a powerful application that simplifies the code execution interception operations, also called “hooking”. Users can now easily monitor and gain control over processes in their systems, to really know what is happening in the Operating System and it’s applications.

With SpyStudio you can monitor and intercept API calls at any time, change its parameters, and resume execution.

SpyStudio uses the Deviare API technology to intercept functions’ calls, this allows the user to monitor and hook applications in real time.
Deviare is a very complex technology, that can be used through the most simple interfaces.

This useful application provides the ability to break process execution and inspect the function’s parameters at any level, and even change its values.

Here is a screenshot of the main window of SpyStudio v1.0.0b, with the new Python console:

Latest improvements on the 1.0.0b version:

• New Python tabbed console allows to handle hooks!
• Python scripts can be loaded from files.
• An initial Python script can be executed on every tab opened.
• New Deviare Database Editor allows to expand the modules and functions database!
• Breakpoint params browser: The return value and the error code are now editable
• Now SpyStudio can run with SeDebugPrivilege enabled or disabled.
• Processes monitoring options are now combinable.
• Select all (Ctrl + A) and Copy (Ctrl + C) options are now available for the output window.
• ‘Filters’ concept changed to ‘Actions’.
• Database expanded: wininet.dll added and winternl.h functions of ntdll.dll added.
• Fixed: Changing a parameter on the params browser made SpyStudio to crash.
• Fixed: Trying to hook a function that was not in the database made SpyStudio to crash when closing.
• Fixed: Changing the ‘Default hook mode’ option was not reflected on the output.

We are glad about how SpyStudio is evolving and we expect users’ reports, comments and suggestions to keep it growing!

Introduction

In this article we analyze Windows Live Messenger 8.5, that is the last version of MSN Messenger.Windows Live Messenger does provide a plugin api (Messenger, Add-ins, and more…) but the features exposed are a very limited subset. Also, this plugin support is not enabled by default and requires the user intervention.Moreover, as of January 3rd 2008 Windows Live Messenger Add-In API this interface is no longer supported.

Hooking

As messenger lacks of a public api we need a way to inject our code in it. There several way to do this, we will be reviewing only two of them.

System wide Hook

We need a hook dll injected in every process of the system. This can be achieved using a CBTProc that creates a system wide window hook. This proc has to check if we are inside Messenger and if it is true, load our dll that contains the code that we want to inject. The advantages of this method are that we are not modifying the installation of Messenger and we are using documented functions, so this approach is likely to work on every version of Windows. On the other hand, our hook dll will be injected in every process and updating requires a reboot that may look suspicious to Anti-Virus software.

Proxy DLL

The idea is to create a proxy dll that will have the same name that other dll used by Messenger. This proxy dll will load our dll when is loaded and will forward all the calls to the original one. In order to avoid renaming or moving system dlls, we can use the trick of placing our dll in the folder where msnmsgr.exe is located so according to Dynamic-Link Library Search Order it will be loaded before the original one placed in system32 folder. The advantage of this method is that affects only Messenger and does not requires to load a dll in every process. The disvantage is that it is associated to a specific version of Messenger.This trick is used by MsgPlus and its dll name is msimg32.dll How was Plus! done?.

Inside MSN Messenger

UI Controls

Previous version of Messenger used standard windows controls to display its UI, but in the latest versions they are using custom controls. Using Spy++ we can see that there is only one window with class name “DirectUIHWND”.

We can see in the picture that this windows lacks of any child window. There was some speculation about they were using windowless controls. These type of controls implement IAccessible interface so if we can obtain such interface we can query the object to look for other interfaces implemented. Unfortunately, most of the interfaces required by the windowless controls are not implemented here. The only interesting interface exported is IOleWindow, but instances of this interface return always the same DirectUIHWND window that is useless.Using Dependency Walker we can check what dlls are used and what is used of each one. One of them is msncore.dll that exports a bunch of C++ functions like DirectUI::NativeHWNDHost::NativeHWNDHost(void). This seems to be the dll used by messenger to display its UI. Further inspection of this dll reveals that it exports DllRegisterServer function. It looks as a COM server, but luckless executing ‘regsvr32 msncore.dll’ from the command line throws an error. After verifying in its resources, we did not find any TLB, so it will require much more research to see if we can get any useful interface from this ’server’.

Resources

The only known way to customize messenger UI is modifying the resources that it uses. These resources are placed in msgsres.dll, that is located in messenger folder. There are lots of sources of information about this type of solution such as How to skin WLM 8.1.This approach can be implemented in two ways: modifing the original file or changing the resource in memory. The first method is easier but has some problems: if the file is updated our changes are lost and antivirus software may warm the user about this modified file. The other method is more complex but lacks of these problems. Resources are usually loaded using LoadResource, LoadImage, etc.; if we can make Messenger get our modified resources instead of the original ones we have reached our goal.

Messenger Plugin Demo

Hooking with Deviare

In order to make Messenger load our modified resources we need to hook the following functions calls: FindResource, LoadResource, LockResource, SizeofResource. We are going to use Deviare API to easily install our hooks.We must know that UI resources are loaded the first time Messenger is loaded. So we need Deviare to notify whenever a new program is started. After creating our SpyManager instance, we can specify that we want to be notified of process creation in this way:

_spyManager->PutReportProcessCreation(_create_process_hook_and_polling, 0)

When OnProcessStarted event is triggered we check if msnmgr.exe was started. If the new process is messenger, we initialize our ResourceManager that installs our hooks. The code that install our hooks is in FunctionHook::Init in the file FunctionHook.h. Only function’s name and function’s module are needed (these are customized using template parameters, in our demo look at DECLARE_FUNCTION_HOOKED).

Managing Resources

ResourceManager receives the events of the hooked functions and modifies the resources we want to customize. It is very simple, it keeps a state of each resource we have to modify, and when requested we return our modified copy.The following call order FindResource → LoadResource → LockResource → SizeofResource is expected for each customized resource.

Adding a Button

We need to add our button to resource 4004:923 (resource type:resource id). This is done when we insert our string.

<Button cmdid=123 id=atom(nktbtn) AccRole=57 Class="TransparentButton" Layout=flowlayout(0,2,0,2)
Active=MouseandKeyboard|NoSyncFocus Padding=rect(5,4,5,4)>
<element class="ToolbarIcon" ID=Atom(ai402)/>
</Button>

To customize the aspect of our button we need to modify resource 4005:923.

Button[id=atom(nktbtn)]
{
accdesc:rcstr(3488);
ShortcutString:rcstr(3489);
AccName:rcstr(3490);
}
element[ID=Atom(ai402)]
{
content:rcimg(3000);
}

Customizing Resources

While customizing resources 4004:923 and 4005:923 is simple because they already exists, returning resources that do not exist is much trickier. Also, strings resources are grouped in 16 strings, check The format of string resources for more info.We customize these resources allocating memory inside Messenger using VirtualAllocEx and returning this address whenever our custom resources are requested. Our hook handler is called after the function fails trying to get invalid resource ids (that are the customized resource addresses). A more robust implementation should place the hook handler before the original function and skip calls when we detect our custom resources are requested.

Enhancements

• Resources are initialized multiple times, when only once will be enough.

Final Result

Demo source code

Requirements

• Visual Studio 2005.
• Windows Live Messenger 8.5.

Source code

Download plugin demo

Services

Nektra offers development services focused in Windows MSN Messenger add-on development

Cookiepie is one of the favorite Firefox extensions for web developers and users of webmail services like Google Gmail, Yahoo and Hotmail.Web developers use cookiepie to test their sites.In the past, if a site (e.g. home banking, community site) required extensive testing simulating many different users, the developer needed to open different browsers. Currently all browsers, including Firefox, have just one place to store cookies.Some users of webmail services have more than one web account to log into. People are using each web account for different purposes but they can’t login to the same site at the same time in the same browser.The Cookiepie Firefox Extension is a solution to this problem, as it allows users to log into different webmail accounts on separate tabs. Try it with two or more of your Gmail, Yahoo or Hotmail accounts.This new release fixes problems experienced in Gmail 2.0, and supports many complex sites like the new Yahoo mail. Even webmail embedded chat is working now!We have made a short video so you can see it in action:

Cookiepie is Free Open Source Software under the GPLv2 license. You can install it now from Nektra Cookiepie site.Please, if you have comments leave them in our group here. We would particularly like to hear about your experience with other sites, such as Facebook. We are making a list of supported websites.

We have released the first version of Deviare. A free trial is available for download.Deviare is a component for ‘easy hooking’ of Windows DLLs. Now you don’t need to be an expert to incercept operating system functions because you use a COM object abstracting many of the complexities.To show the power look at the following code snippet in CSharp (.NET):

DeviareTools.IProcesses procs = _mgr.get_Processes(0);
DeviareTools.IProcess proc = procs.get_Item("msnmsgr.exe");
DeviareTools.IPEModuleInfo mod = proc.Modules.get_ModuleByName("ws2_32.dll");
DeviareTools.IExportedFunction fnc = mod.Functions.get_ItemByName("send");
hook = mgr.CreateHook(fnc);
hook.Attach(proc);
hook.OnFunctionCalled += new Deviare.DHookEvents_OnFunctionCalledEventHandler(hook_OnFunctionCalled);
hook.Properties = (int)DeviareCommonLib.HookFlags._call_before;
hook.Hook();
void hook_OnFunctionCalled(DeviareTools.Process proc,DeviareParams.ICallInfo callInfo, Deviare.IRemoteCall rCall)
{
    DeviareParams.IParams pms = callInfo.Params;
    DeviareParams.IEnumParams enm = pms.Enumerator;
    DeviareParams.IParam pm = enm.First;
    pm = enm.Next;
    object[] args = new object[1];
    string msg = "Transmition -> ";
    msg += pm.Value;
    msg += "\r\n";
    args[0] = msg;
    txtOutput.Invoke(new AppendHandler(Append), args);
}

With this simple code you hook the send function in the WinSock dll for the Messenger process and our own function hook_OnFunctionCalled is called before the ‘real send’The code can be written in any COM friendly programming language like: C++, C#, VB, Java, Python, Perl, Ruby and many others. API Hook examples in C++, C#, VB.Many applications can now be built on Deviare Technology like Spy Studio a tool to monitor Windows API and available for free.

As you may already know, Nektra core skills & knowledge can be briefly summarized in system internals & problem solving, innovation and creativity. We were working hard to introduce new products in the market and it’s very stimulating when you receive “Wows!” from people who can appreciate the complex stuff we have made.

Our next play is Spy Studio, Deviare, OEAPI for Vista & NKTWAB license change to LGPL:

Spy Studio is a new tool for hooking microsoft windows applications, it has an intuitive interface and you don’t need to be an expert in assembler or reverse engineering to insert hooks into different API’s or DLL’s. It has many interesting applications like seeing what your software is doing internally, and from the business perspective it’s very useful for monitoring, isolating processes access to some API or DLL, debugging, litigation & software forensics, support, software engineering blackbox testing, etc. You can download it now from here. We would be very glad to receive you comments in our forums

Deviare is the component to do your own applications and the framework used to develop Spy Studio. You can do your own hooking application with it, and extends Spy Studio’s possibilities to your own requirements, for example your own api monitors, administration tools, themes/skins/gui (i.e: scrollbar issues), posture agents, intrusion detection at the application level, etc.

OEAPI has been growing and now supports Vista’s Windows Mail in addition to Outlook Express. We are currently at the version 3.1.2 and 3.2.0 will be released very soon. OEAPI has really improved in demos, documentation, performance and capabilities. There is an updated list at What’s new section.

NKT WAB is now LGPL and it shows how to implement features not available or documented in the microsoft windows api. This component is useful for accessing the WAB (Windows Address Book), creating groups & folders and now supports Vista’s contacts too.

There is new stuff coming, but the most important thing is that our customers continue expressing their Wows!