Category Archives: SpyStudio

64-bit Microsoft Office Applications Do Not Like Stack Walking

Our SpyStudio tool intercepts application system calls and retrieves the called functions by inspecting the call-stack.

We were unable to access stack information in the last few 64-bit releases of Microsoft Office products. When we use SpyStudio to intercept an Office installation or most Office applications like Word or Excel, they start normally, but eventually exit silently.

Our initial research showed the culprit to be the Office Software Protection Platform, which would make sense, as it is supposed to hide an application’s safety mechanisms.

However, the Office Software Protection Platform did not keep us from intercepting the 32-bit version of Microsoft Office, so we decided to investigate further.

The .pdata Section

How does the 64-bit operating system do a stack walk when an exception occurs, and how can a debugger know who called a function?

Microsoft added new metadata information  which is stored on a special section named “.pdata” in the PE file format specification. When an application is compiled, the compiler stores information related to the prolog of each function. If a function handles exceptions, the compiler also stores data about actions that must happen when the unwind operation is executed.

The operating system and the debugger use a series of RUNTIME_FUNCTION structures to retrieve a variety of information about each function, like how much stack space is reserved for each function usage, which callbacks must be called in an unwind operation, and where assembly registers are stored.

If you want to do your own stack walking, you can start from the current program counter (the RIP register in x64) and look for the RUNTIME_FUNCTION that belongs to it. Then process the UNWIND_INFO items to determine stack usage, and lastly, retrieve the location of the return address of the parent function.

Fortunately, there are some new APIs which makes the job easier. They are RtlLookupFunctionEntry and RtlVirtualUnwind. A stack walking code sample:

VOID StackTrace64(VOID)
{
    CONTEXT Context;
    KNONVOLATILE_CONTEXT_POINTERS NvContext;
    UNWIND_HISTORY_TABLE UnwindHistoryTable;
    PRUNTIME_FUNCTION RuntimeFunction;
    PVOID HandlerData;
    ULONG64 EstablisherFrame, ImageBase;

    DbgPrint("StackTrace64: Executing stack trace...\n");
    // First, we'll get the caller's context.
    RtlCaptureContext(&Context);
    // Initialize the (optional) unwind history table.
    RtlZeroMemory(&UnwindHistoryTable, sizeof(UNWIND_HISTORY_TABLE));
    UnwindHistoryTable.Unwind = TRUE;
    // This unwind loop intentionally skips the first call frame, as it shall
    // correspond to the call to StackTrace64, which we aren't interested in.
    for (ULONG Frame = 0; ; Frame++)
    {
        // Try to look up unwind metadata for the current function.
        RuntimeFunction = RtlLookupFunctionEntry(Context.Rip, &ImageBase,
                                                 &UnwindHistoryTable);
        RtlZeroMemory(&NvContext, sizeof(KNONVOLATILE_CONTEXT_POINTERS));
        if (!RuntimeFunction)
        {
            // If we don't have a RUNTIME_FUNCTION, then we've encountered
            // a leaf function.  Adjust the stack approprately.
            Context.Rip  = (ULONG64)(*(PULONG64)Context.Rsp);
            Context.Rsp += 8;
        }
        else
        {
            // Otherwise, we call upon RtlVirtualUnwind to execute the unwind
            // for us.
            RtlVirtualUnwind(UNW_FLAG_NHANDLER, ImageBase, Context.Rip,
                             RuntimeFunction, &Context, &HandlerData,
                             &EstablisherFrame, &NvContext);

        }
        // If we reach an RIP of zero, this means that we've walked off the
        // end of the call stack and are done.

        if (!Context.Rip)
            break;
        // Display the context
        DbgPrint("FRAME %02x: Rip=%p Rsp=%p Rbp=%p\n", Frame, Context.Rip,
                 Context.Rsp, Context.Rsp);
    }
    return;
}

Source: Programming against the x64 exception handling support, part 7

Because the .pdata section is created when the application is compiled, if the program generates dynamic code, like the .NET JIT profiler does, it should also create the corresponding RUNTIME_FUNCTION metadata and inform the operating system of the new dynamic code.

The RtlInstallFunctionTableCallback API adds an entry in an internal processes table maintained by NtDll.dll that helps RtlLookupFunctionEntry and RtlVirtualUnwind find information on how to walk the stack when your dynamically generated code is in the middle of the function calls chain.

Microsoft Office Installer and Applications

So what explains the silent exit of an Office application like Word?

At first we thought that some kind of intentional data corruption was happening in the internal table. It seemed like some kind of anti-debugging technique to keep reverse engineers from seeing the product activation mechanism.

After some trial and error we noticed that RtlInstallFunctionTableCallback was installing a callback to a suspicious routine. Surprisingly, that routine calls TerminateProcess API! When our stack walker function wanted to know the chain of calls, RtlLookupFunctionEntry indirectly called that routine and the program terminated silently.

Could the guys at Microsoft have decided to use this strange method to protect their code? The annoying callback function was added and removed frequently during many operations.

The solution was to add a simple hook to the RtlInstallFunctionTableCallback API to keep it from being added. Now the issue is resolved and the fixed versions of Deviare “Hooking for the Masses” and SpyStudio will be available soon.

If you liked this article, you might also like:

Nektra and VMware are Collaborating to Simplify Application Virtualization Packaging

Nektra’s SpyStudio provides tools allowing for application harvesting and simplified packaging for the VMware ThinApp offering

Nektra and VMware, Inc. have been collaborating to significantly improve the process of creating and troubleshooting VMware ThinApp application virtualization packages with the SpyStudio application. The video below shows the SpyStudio new ability to harvest applications directly from the operating system to create VMware ThinApp packages even for applications where customers no longer have the installer media. The new SpyStudio release can be downloaded free from http://www.nektra.com/products/spystudio-api-monitor/download/. A fully featured version requires a license.

As companies embrace application virtualization, packaging and troubleshooting in some cases has proven difficult and resource intensive. SpyStudio offers a means to quickly compare application operations to help pinpoint and solve challenging packaging tasks.

The application virtualization market will continue to grow in the next years. Hence, it is critical to be able to make the virtualization process less error prone. Leveraging SpyStudio, a unique and advanced tool will prove to be an essential part of the growth of application virtualization and successful customer implementations.

If you liked this article, you might also like:

  1. Case Study: Nektra Escalation Support for Symantec Workspace Virtualization (SWV)

Resources

Benchmarking Microsoft Office’s PowerPoint Virtualization: Microsoft App-V Vs. Symantec Workspace Virtualization

Introduction

Benchmarking virtualization products is an addictive game. Below we use SpyStudio to compare how long it takes to load a 21 slides PowertPoint presentation, with text and graphics, in three environments: Microsoft App-V 5 (MNT), Symantec Workspace Virtualization 6.4.1603, and natively. We measure the time elapsed until the slides show up on the GUI.

Both end-users and developers benefit from the virtualization of Microsoft Office applications. One benefit for end-users is that the virtualization of Microsoft Office does not leave the footprints that the installed program would: Microsoft Office natively installs a lot of components on the user machine that impact on their performance. Developers benefit from the chance to develop add-ins for different versions of the same application without having to run different virtual machines. For example, it is possible to develop an extension for Outlook and run it on Outlook 2007, 2010, and 2013 on a single machine.

Microsoft has been encouraging the virtualization of Microsoft Office. They are still deploying the usual Office applications but they are also introducing App-V packages for them. Other virtualization companies are following suit.

Benchmark

NativeApp-VWorkspace Virtualization
Min31.5137.2634.33
Max48.7351.5440.71
Avg36.7741.6937.52
Median35.3540.3137.52
SD5.424.301.80

Conclusions

In general, the overhead for this benchmark is low. Microsoft App-V is just 12% slower, while Symantec Workspace Virtualization is only 2 % slower. Symantec wins again.

See Also

  1. Application Virtualization Testing and Troubleshooting with Spy Studio and Deviare
  2. Benchmarking IE6 Virtualization: VMWare ThinApp vs. Symantec Workspace

Benchmarking IE6 Virtualization: VMware ThinApp vs. Symantec Workspace Virtualization

Introduction

Below we use SpyStudio to compare ThinApp and Workspace Virtualization performance. Both Symantec and VMware highlight the use of application virtualization to run legacy web applications. There is a huge number of mission critical web applications that only run correctly on Internet Explorer 6 and while companies may be able to afford the cost of migrating applications to modern browsers, they cannot afford even a short application interruption. Virtualization allows companies to continue to run their legacy applications while moving to more modern technology.

The challenges of virtualizing IE6 are not limited to rendering HTML: Java applets, ActiveX, and Flash must also be virtualized. This benchmark compares how long it takes to: launch Internet Explorer 6, launch IE6 and navigate to Nektra’s blog, and open Internet options. We ran each test ten times. SpyStudio can also be used to benchmark specific bottlenecks in plugins and components and to identify compatibility issues.

Benchmark

Launch IE6

Symantec Workspace VirtualizationVMWare ThinApp
Min1.5730167 secs2.217207 secs
Max1.800602 secs2.5048638 secs
Avg1.681593789 secs2.360678356 secs
Median1.6773198 secs2.3479585 secs
SD0.076693291 secs0.104027481 secs

Launch IE6 and Navigate to http://blog.nektra.com

Symantec Workspace VirtualizationVMWare ThinApp
Min7.0385753 secs9.0458817 secs
Max7.3155127 secs
10.1313509 secs
Avg7.152178211 secs
9.4426123 secs
Median7.1880917 secs9.3928642 secs
SD0.098509252 secs0.293977928 secs

Open Internet Options

Symantec Workspace VirtualizationVMWare ThinApp
Min1.8866595 secs2.5610158 secs
Max2.215396 secs2.9904516 secs
Avg2.025388122 secs2.767548167 secs
Median2.0197439 secs2.7876525 secs
SD0.101490761 secs0.152657664 secs

Conclusions

These benchmark results show that Symantec Workspace Virtualization is faster in all three tests.

Symantec Workspace Virtualization was an average of 71 % faster when launching IE6, 75 % faster when launching IE6 and opening Nektra’s blog, and 73 % faster when launching IE6 and opening Internet options.

You can easily benchmark these and other virtualization products yourself with SpyStudio. Download a free trial here.

Related Services

  1. Application Virtualization and Packaging
  2. Interception and Filter Drivers Services

See Also

  1. Application Virtualization Testing and Troubleshooting with Spy Studio and Deviare
  2. Benchmarking Microsoft Office’s PowerPoint Virtualization: Microsoft App-V Vs. Symantec 

 

Injecting a DLL in a Modern UI Metro Application

Dll injection is one of the oldest techniques used to run custom code inside a target application in Windows. It is usually used to intercept and modify normal application behavior or add new functionality.

Injecting a DLL in a target process is a relatively easy task: you simply create a remote thread that calls LoadLibrary using CreateRemoteThread or NtCreateThreadEx. You will need some privileges to be able to access the injected process but that is beyond the scope of this article.

When you try to inject a library into a Windows 8 Modern UI Metro application you will find that although the injection code works as expected, your DLL will NOT load, LoadLibrary will return FALSE and GetLastError will return ERROR_ACCESS_DENIED.

Well, you think… Modern UI applications have very limited access to computer resources and run in a sandboxed environment, so problems are to be expected.

While doing some research on how to add new functionallity to the Windows Mail application that comes with Windows 8 and how to hook Modern UI apps using Deviare, we needed find out why LoadLibrary was failing.

Reverse engineering comes into play

We started to analyze what LoadLibrary does. It calls LoadLibraryEx with dwFlags=0 and LoadLibraryEx does some checks. First stop.

If you want to load a package you must use the LoadPackagedLibrary API. If you want to load a normal DLL, you have to use LoadLibrary[Ex]. LoadPackagedLibrary documentation says that the path cannot be absolute or contain “..” but these checks are mainly done in the LoadLibraryEx routine. The only difference between LoadLibrary and LoadPackagedLibrary is wether the dwFlags parameter has a value of 4 or of zero.

Among other things, LoadLibraryEx will build the search path to locate your DLL and then call the undocumented LdrLoadDll. Because we want to enforce using the path where our library is located, we changed our code to call LdrLoadDll directly.

Second Try:

Although LdrLoadDll correctly found our dll, when we used SpyStudio to check for errors, we saw that a call to NtOpenFile failed, reporting STATUS_ACCESS_DENIED. We realized that there was a security-related issue.

Using the icacls.exe utility, we set the DLL file privileges to allow read and execute access to low integrity processes. Also we added a special Windows 8 user named “ALL APPLICATION PACKAGES” to the list of users with permission to read and execute the DLL code.

Third Try:

NtOpenFile passed initial security checks, but our DLL was still not loading.

Continuing to research LdrLoadDll, we jumped into the kernel-mode of the “NtCreateSection” API and got that the “CiValidateImageHeader” function of ci.dll was returning  a STATUS_INVALID_IMAGE_HASH error so we added a digital signature to the file. To prevent future problems, we used a real certificate instead of the self-signed one.

Now CiValidateImageHeader was ok, but a later call to “CiValidateImageData” returned the same error. We then added the “/ph” parameter to the SignTool.exe utility to include pages hashes in the signing process.

Fourth Try:

Well, we thought: we have a signed dll, privileges are ok. Let’s try again.

FAIL.

This time, the culprit was a function named “SeGetImageRequiredSigningLevel” located in ntoskrnl.exe. SeGetImageRequiredSigningLevel checks the minimum certificate requirements to load a DLL inside a WinRT application.

We realized we needed to sign our DLL with a cross-certificate, like those used to sign kernel-mode drivers.

Conclusion:

We could not continue our tests because we do not have that kind of certificate right now, but we discovered that a kernel setting determines what kind of certificate is checked by SeGetImageRequiredSigningLevel.

This blog post explains how to manually bypass the security check and run untrusted applications on a Microsoft Surface device using WinDbg. We can follow a similar procedure to manually bypass the security check and correctly map and inject the DLL in the WinRT application on the desktop Windows operating system.

The exception:

Before starting our whole research, we already had a method for injecting a DLL in WinRT applications: copy the DLL file inside the System32 folder and voilá! Although you need administrative privileges to copy a file to the System32 folder, once there, you can load it using LoadLibrary without using a path, since this folder is one of the default locations the Windows operating system will search. In addition, because you are using a relative-path, some security checks are skipped. Another plus is that you will not ever need to digitally sign the file!

But like many companies, we want to avoid copying DLL files into the ever-growing System32 folder and keep our files in the same location as our application. This is why we started our research.

Related Services

  1. Reverse Engineering
  2. Interception and Filter Drivers Services
  3. Application Virtualization and Packaging

Application Virtualization Testing and Troubleshooting with SpyStudio and Deviare

In Spy Studio: Solving an Issue on IE6 we troubleshot an issue that happens while virtualizing an Internet Explorer 6 (IE6) layer for Japanese using Symantec Workspace Virtualization. IE6 worked when IE8 was installed as the base operating system but not when the base was IE9. In the following screenshot we show how Spy Studio’s trace compare function analyzes the difference between running IE6 with the IE8 and with the IE9 base on the Windows 7 operating system.

As you can see, this user interface is much more informative than comparing a long list of line-oriented logs.

Imagine that an App-V Package does not work as expected. Can it be solved with the App-V Sequencer alone or do you need more advanced tools? Or what if the IT department tries to simplify the process of application virtualization with VMware ThinApp Factory, while the virtualization team tries to pinpoint an issue with Process Monitor? Comparing long logs is tedious and error prone. How can testing and troubleshooting be improved? These common issues extend to all application virtualization products, including: Symantec’s Workspace Virtualization, Spoon, and Cameyo.

Spy Studio and Deviare can both be used to troubleshoot virtualization. Spy Studio’s graphic console allows you to record application behavior and compare different runs in different environments. You can compare the differences between running a fully virtualized application and running it in a physical environment. Spy Studio complements Process Monitor, and can also import Process Monitor’s logs for comparison. In Comparing Traces with Spy Studio we provide a step by step example of how to compare traces.

Deviare is a more advanced tool which enables application virtualization professionals and QA engineers to build their own custom troubleshooting software.

If you are interested in learning how to develop and customize your own registry monitor visualization tool please take a look at Instrumenting Instrumenting Binary Applications with VBScript and Deviare.

Resources

  1. Application Virtualization Solution Overview and Feature Comparison Matrix
  2. Application Virtualization Troubleshooting and Support
  3. APP-V Troubleshooting … Demystified
  4. VMWare ThinApp Troubleshooting Blog
  5. P2V

Related Services

  1. Reverse Engineering
  2. Interception and Filter Drivers Services
  3. Application Virtualization and Packaging

See Also

  1. Benchmarking IE6 Virtualization: VMWare ThinApp vs. Symantec Workspace Virtualization
  2. Benchmarking Microsoft Office’s PowerPoint Virtualization: Microsoft App-V Vs. Symantec Workspace Virtualization

The truth about Google Chrome using Spy Studio

Everyone has a lot of questions about Chrome.  Some people say that it is spyware because each and every character you enter is sent to Google.  Hundreds of comments like this can be found on the web, like this one that says “Chrome spends nearly as much time phoning home to Google as it does talking to other Web servers.”  On the other hand, you can also find on the web the opposite opinion that claims “If you do not wish this data to be sent to your search provider, you have a number of options: Use incognito mode, turn off search suggestions permanently or change your search provider.”

Who is correct?  What kind of information is really traveling between Chrome and Google?  What data about you is being sent to the web?  Is it true that Google’s browser sends details about everything you do?  Is it an unsafe browser?  What happens behind Incognito mode?

The first thing we want to know is “What information does Chrome send about visited sites to Google”? Many different opinions can be found on the web, and some are really alarming.  One person says that toolbarqueries.google.com collects everything the browser sends to it.  This is indeed true, and you can see in metrics_service.cc [chromium.org], what information about visited websites is being sent.  Although this only happens if you selected it in Chromes ‘Under the Hood’ (Options -> “Help make Google Chrome better by automatically sending usage statistics and crash reports to Google”) this option is not selected by default, you have to specifically select it during the Chrome installation.  Using SpyStudio you can be 100% certain about this by checking and un-checking the option, and watching all the ‘send’ function calls.  So, does Google Chrome send information about every website you visit to toolbarqueries.google.com?  The answer is no, it does it only if you request it to.  This doesn’t mean that other information, like the one send to google-analytics, is not being sent anymore.

However it is interesting to notice that this behavior is exactly the same under Incognito mode.  This means that if the option of sending usage statistics is checked, it doesn’t matter what mode Chrome is running, the statistics are sent anyway.  We know that the only differences between normal and Incognito modes are the logging of websites visited, files downloaded, download histories and cookies.  So this feature is local to the machine, and nobody has said that statistics are not sent under this mode.  Although I think for many of us, we implicitly assume to be anonymous while running Chrome under Incognito mode.  So we better keep the limitations of this feature in mind!  Again, this only applies when sending statistics option is selected.

The other feature we want to inspect is the suggestion made by the address bar: “When you type URLs or queries in the address bar, the letters you type are sent to Google so the Suggest feature can automatically recommend terms or URLs you may be looking for.”  This is highly controversial, we want to know about this feature when using Incognito mode (in which the suggest feature seems to be automatically disabled). Again we can use SpyStudio to make sure.  You can see that Chrome does not send any information to Google about your key strokes when using Incognito mode.  You can also watch calls to GetAddrInfoW function, which provides protocol-independent translation from a Unicode host name to an address.
When you are not running on Incognito, you can turn this off by right clicking on the address bar and selecting “Edit search engines…” Then uncheck the check box at the bottom labeled “Use a suggestion service to help complete searches and URLs typed in the address bar”.

We can now safely stop all the paranoia about Chrome.  We can see the information that Google Chrome sends to Google using SpyStudio and we know that this depends on the options you choose.  So Chrome is not spyware that sends everything you do to Google.   I also believe it is important to understand what features the Incognito mode provides and not assume things about it.

Watch Google Chrome

See for yourself the information that Google Chrome sends to Google.  Use Nektra’s SpyStudio to monitor Chrome’s behavior.  It is very easy:

  1. Download SpyStudio from Nektra’s website free of charge and install it.
  2. Replace the database ‘deviare.fdb‘ with a new version.  You will find ‘deviare.fdb’ in the path you installed SpyStudio: SpyStudiobin
  3. Download the script chromewatcher and then add the path where you saved it to SpyStudio.  Edit -> Preferences -> Python
  4. Run SpyStudio and import the module chromeWatcher by typing “import chromeWatcher” in the Python console.  Then start monitoring by calling the Begin() function by typing “chromeWatcher.Begin()”.
  5. Now watch SpyStudio while using Google Chrome to find out what information is sent by Chrome.

What does the ChromeWatcher script do?

The ChromeWatcher module was specially made to capture calls to the Winsock functionssend‘ and ‘WSASend‘. To know where the information is going, a socket connections track must be kept.  So it is necessary to hook ‘connect’ and ‘select’ functions too.  The idea behind ChromeWatcher is to hook ‘send’ and ‘WSASend’ calls that are made to Google and show them to you.
To understand better this script you can see SpyStudio documentation on: SpyStudiodoc

GoogleToolbar PageRank requests

Using our API hooker SpyStudio I wrote a script to intercept http requests done using wininet.dll API coming from a specific module of a process. The script keeps request information (server and url) to display in next calls and let filter requests to a specific server. Its name is httpReport.py and can found in SpyStudio v1.0.1 distribution.

httpReport navigates the stack in each call to wininet.dll functions to see what module called the hooked function, filtering all modules except the specified. This feature and server name filtering, allow a fine interception.

To use the script keep only one instance of iexplore.exe (the script will only hook the first instance if there are more than one) and type these lines in SpyStudio python console:

import httpReport
httpReport.startIe(‘toolbarqueries’, ['googletoolbar2.dll'])

The script will display queries done to a server that contains the string ‘toolbarqueries’ coming from module ‘googletoolbar2.dll’.

For example, if TechCrunch page is inserted in the address bar we get a wininet.dll!InternetConnectA call to ‘toolbarqueries.google.co.uk’ server and then a GET request to this url:

/search?client=navclient-auto&googleip=O;64.233.169.147;266&iqrn=ZjbD&orig=0PnmJ&ie=UTF-8&oe=UTF-8&features=Rank:&q=info:http%3a%2f%2fwww%2etechcrunch%2ecom%2f&ch=751153802320

There are some parameters that need more research to be understood but there are some others we can tell something:

googleip: indicates Google server used for the query

ie: iexplore encoding?

oe: maybe Outlook Express encoding?, only a bad guess

features: what we are asking to the server (here ‘Rank’)

q: encoded url (http%3a%2f%2fwww%2etechcrunch%2ecom%2f = http://www.techcrunch.com/)

ch: it looks as a function to the url to prevent other client to do the same requests

Then, wininet.dll!InternetReadFile return the http response (to see it enable the option ‘Show Params on Return’ in Preferences):

‘Rank_1:1:8n’

that indicates that the page visiting has PageRank 8.

This process is repeated for every page you visit so Google can collect all the pages browsed by all the users using GoogleToolbar. That’s why it may be considered as a spyware.

SpyStudio 1.0.0b released!

Introduction to SpyStudio:

SpyStudio is a powerful application that simplifies the code execution interception operations, also called “hooking”. Users can now easily monitor and gain control over processes in their systems, to really know what is happening in the Operating System and it’s applications.

With SpyStudio you can monitor and intercept API calls at any time, change its parameters, and resume execution.

SpyStudio uses the Deviare API technology to intercept functions’ calls, this allows the user to monitor and hook applications in real time.
Deviare is a very complex technology, that can be used through the most simple interfaces.

This useful application provides the ability to break process execution and inspect the function’s parameters at any level, and even change its values.

Here is a screenshot of the main window of SpyStudio v1.0.0b, with the new Python console:

SpyStudio v1.0.0b Main Window

Latest improvements on the 1.0.0b version:

  • New Python tabbed console allows to handle hooks!
  • Python scripts can be loaded from files.
  • An initial Python script can be executed on every tab opened.
  • New Deviare Database Editor allows to expand the modules and functions database!
  • Breakpoint params browser: The return value and the error code are now editable
  • Now SpyStudio can run with SeDebugPrivilege enabled or disabled.
  • Processes monitoring options are now combinable.
  • Select all (Ctrl + A) and Copy (Ctrl + C) options are now available for the output window.
  • ‘Filters’ concept changed to ‘Actions’.
  • Database expanded: wininet.dll added and winternl.h functions of ntdll.dll added.
  • Fixed: Changing a parameter on the params browser made SpyStudio to crash.
  • Fixed: Trying to hook a function that was not in the database made SpyStudio to crash when closing.
  • Fixed: Changing the ‘Default hook mode’ option was not reflected on the output.

We are glad about how SpyStudio is evolving and we expect users’ reports, comments and suggestions to keep it growing!