VTBL is an IDA script which identifies all the virtual tables found in any module of a native process. The virtual tables can be related to a COM or a C++ class. Unlike other tools, ours does not depend on a specific compiler to obtain a virtual table. This makes it an essential tool for reverse engineers.

The script works on all IDA versions. To use it, you must:

  1. Use IDA to disassembly the module you want to analyze.
  2. Load the “VTBL.IDC” script from File -> Script File or by using the ALT F7 shortcut.
  3. VTBL.EXE will be executed.
  4. Select the process you want to analyze.
  5. Enable the suspension of the process if you want to intercept the process from the beginning.
  6. Select the module you want to analyze. It must be the same module you disassemble in step one.
  7. Once the analysis is over, select the virtual table to obtain its cross reference. The tool displays the number of functions the virtual table contains.
  8. Hook the selected virtual table
  9. Close the VTBL.EXE dialog
  10. Both, the disassembled code and the IDA output window will display all processed cross references.

We tried the tool out on Notepad++.exe. See video below. We used open source software so we could compare the results with the original source code.

We ran Notepad++.exe, selected the Notepad++.exe module and waited until all the virtual tables had been analyzed. The tool displays a list of virtual tables with the following format: VTBL_X1_X2_X3, where X1 is the index, X2 the start address, and X3 the function count. We hooked the virtual table with “CD” in the index field.

After the process we closed VTDL.EXE and analyzed the results with IDA Pro.

Prerequisites

  1. Deviare Hooking Engine
  2. Compile the VTBL_Code\Helper\Helperhelper.vcproj
  3. Open the Visual Studio 2010 project
  4. Change the Form1.cs DLL imports to point to the helper.dll and DeviareCOM.dll
  5. Compile the project
  6. Open VLTB.idc and modify the full paths of DeviareTest.exe and CrossRef.dat

Source Code

VTBL is available as vtbl-ida-pro-plugin.

See Also

  1. Injecting a DLL in a Modern UI Metro Application
  2. Reverse Engineering and The Cloud

In Spy Studio: Solving an Issue on IE6 we troubleshot an issue that happens while virtualizing an Internet Explorer 6 (IE6) layer for Japanese using Symantec Workspace Virtualization. IE6 worked when IE8 was installed as the base operating system but not when the base was IE9. In the following screenshot we show how Spy Studio’s trace compare function analyzes the difference between running IE6 with the IE8 and with the IE9 base on the Windows 7 operating system.

As you can see, this user interface is much more informative than comparing a long list of line-oriented logs.

Imagine that an App-V Package does not work as expected. Can it be solved with the App-V Sequencer alone or do you need more advanced tools? Or what if the IT department tries to simplify the process of application virtualization with VMware ThinApp Factory, while the virtualization team tries to pinpoint an issue with Process Monitor? Comparing long logs is tedious and error prone. How can testing and troubleshooting be improved? These common issues extend to all application virtualization products, including: Symantec’s Workspace Virtualization, Spoon, and Cameyo.

Spy Studio and Deviare can both be used to troubleshoot virtualization. Spy Studio’s graphic console allows you to record application behavior and compare different runs in different environments. You can compare the differences between running a fully virtualized application and running it in a physical environment. Spy Studio complements Process Monitor, and can also import Process Monitor’s logs for comparison. In Comparing Traces with Spy Studio we provide a step by step example of how to compare traces.

Deviare is a more advanced tool which enables application virtualization professionals and QA engineers to build their own custom troubleshooting software.

If you are interested in learning how to develop and customize your own registry monitor visualization tool please take a look at Instrumenting Instrumenting Binary Applications with VBScript and Deviare.

Resources

  1. Application Virtualization Solution Overview and Feature Comparison Matrix
  2. Application Virtualization Troubleshooting and Support
  3. APP-V Troubleshooting … Demystified
  4. VMWare ThinApp Troubleshooting Blog
  5. P2V

See Also

  1. Benchmarking IE6 Virtualization: VMWare ThinApp vs. Symantec Workspace Virtualization
  2. Benchmarking Microsoft Office’s PowerPoint Virtualization: Microsoft App-V Vs. Symantec Workspace Virtualization

The Deviare Interception Engine includes a feature that allows developers to add plugins to hooks. When a hooked API is called, it will raise the OnFunctionCalled method of all attached plugins to allow the reading and writing of parameters, and passing custom parameters to the INktSpyMgr object. Since the method is called in the context of the hooked application, the plugin will have local access to all process data.

We use COM as the bridge between the plugin code and the internal engine to support plugin development in C/C++ and any other .NET language. To use COM, the programmer must either register DeviareCOM libraries using RegSvr32 or use Registration Free COM.

The second method is best as it avoids registering components on the computer and asking for elevated privileges to accomplish the task. However in order to enable RegFree-COM, you will need to access the hooked executable manifest file, which is usually embedded in the application, and modify it, which may infringe copyright and have undesirable side effects. In addition, you cannot attach a manifest file to Deviare’s agent module: it will simply be ignored because the OS loader only parses executable manifests, and only on startup.

When a hooked api is called, an INktHookCallInfoPlugin object is created and passed to the plugins to let them know the context of the api call. But, how did we solve the management of COM objects without registering COM classes nor using a manifest file? Here is where Activation Context API comes in.

Activation context API lets us create, activate and manage activation contexts in the same way Windows loader does. First we embed our manifest file as a resource inside DvAgent.dll. Once the agent is attached to the hooked process, it creates an activation context using the data of the embedded resource.

When a hooked API is called, we must:

  1. Activate our custom context.
  2. Create the INktHookCallInfoPlugin object.
  3. Call the plugins callbacks (letting them manage all DeviareCOM objects as they want)
  4. Deactivate the custom context and continue execution.

Our benchmark shows that performance penalty in activating/deactivating contexts is minimal. If you need additional help using registration free COM please do not hesitate to ask in our forum.

See Also

  1. Improving Deviare Hooking Performance with Custom Hooks

More Binary Instrumentation Alternatives

Deviare now has custom hooks to improve hooking performance. With custom hooks, two “OnFunctionCall” events are triggered: one in the SpyMgr process, and the other within the agent. The custom hook can send data, or custom parameters. to the SpyMgr process. The custom Deviare modules running in the agent have access to all process data, including file and registry handles. Since custom hooks access local process data, it is now possible to get the full key path of a registry HKEY value and transmit the resulting string as a custom parameter instead of doing a “DuplicateHandle” call in the SpyMgr process. You can also combine asynchronous hooking with custom hooks to keep the “OnFunctionCall” event in SpyMgr from blocking the flow of the application.

The Deviare Hooking Engine aims to simplify the “hooking experience” for people who are outside the Windows internals, x86/x64 Assembly, and reverse engineering fields, there is usually a penalty when you want to abstract something complex like API function hooking in a developer friendly COM component like Deviare. Our custom hooks are designed to meet different needs.

Another interesting advantage of using custom hooks is that you can mix different programming languages. For example, you can add a custom hook in C++ but notify the SpyMgr in C#, or vice versa.

The sample code below monitors and intercepts file mappings via the function MapViewOfFile on notepad.exe 32 and 64 bit. The custom hook analyzes the buffer in search of a specific string posing as malware. The read is cancelled if that string is found in the buffer. The sample code has two modes to illustrate the difference between only using SpyMgr, and relying on custom hooks: the displayed combo switches from one mode to the other. Custom hooks are also essential for video and audio processing. If you are interested in bootstrapping a project that requires audio hooking, do not hesitate to look at our Audio Recorder API. It will save you a lot of time and money. This code also hooks both 32 and 64 bit process from the same (“Any CPU”) assembly.

Code

The code is available on github.

Prerequisites

  1. Install the latest Deviare version
  2. Register 32 and 64 bit COM DLLs (although registration-free COM can be used in custom hooks too)
  3. Download the source code
  4. Copy Nektra.Deviare2.dll to the Lib directory inside the project
  5. Build the Debug configuration
  6. Run Notepad
  7. Run the hooking console

 

One of the top uses of our Windows Live Mail API is integrating a vendor antivirus to the Windows Live Mail desktop client. For example, Trend Micro’s Titanium Security Solutions uses it to integrate with Windows Live Mail and recognize viruses. The C# code sample below uses the nClam library to interface with an open source project: Clam AntiVirus.

While integrating Windows Live Mail API with an antivirus engine helps to mitigate risks from e-mails there are obviously many other sources of security risks. Exploits came in many disguises: they can take advantage of vulnerabilities of web browsers or of network services running on your machine. The Deviare Interception Engine can unmask them. It intercepts network APIs and can be used to discover and deactivate malware at the network level, not just within a specific application.

The code can be modified to work with commercial antivirus engines such as those by Symantec, Trend Micro, McAfee, Kaspersky, F-Secure, ESET, AVG, Microsoft, Bitdefender, and GFI. Since the list is long and each of those APIs has different idiosyncrasies, companies such as OPSWAT have developed a unified API. The various “Malware Analysis Online Services” can also be easily integrated to a custom solution.

Code

Prerequisites

  1. Visual Studio 2010
  2. Request a trial for Windows Live Mail API here
  3. Download ClamAV here and VC redistributables if needed
  4. Extract the Clam Anti Virus binaries to a specific directory and create an empty folder called DB there
  5. Run CMD as Administrator, go to the ClamAV directory and run freshclam.exe
  6. Install the ClamAV service by running: clamd –install
  7. Install the ClamAV update service by running: freshclam –install
  8. Download nClam from here and add/modify it as a reference in the project
  9. Get the Windows Live Mail API Anti-Virus example from github

Main.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using Microsoft.Win32;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using NktWLMailApi;
using NktWLMailApiInit;
using NktWLMailStore;
using nClam;

namespace Antivirus
{
    public class Main
    {
        static string pluginsKey = "Software\Nektra\WLMailApi\Plugins";
        static string pluginValueName = "AntivirusDemo";
        static string pluginValueData = "Antivirus.Main";
        private NktWLMailApiInit.WLMailApiInit wlmailApiInit;
        private NktWLMailApi.WLMailApi wlmailApiCore;
        private NktWLMailStore.FolderManager folderManager;
        private NktWLMailStore.MailAccountManager accountManager;
        private Utils utils;
        private nClam.ClamClient clamClient;
        private const string CLAM_HOST = "localhost";
        private const int CLAM_PORT = 3310;

        [ComRegisterFunctionAttribute]
        public static void RegisterFunction(Type t)
        {
            RegistryKey key = Registry.LocalMachine.OpenSubKey(pluginsKey, true);
            if (key == null)
            {
                key = Registry.LocalMachine.CreateSubKey(pluginsKey);
                if (key == null)
                {
                    System.Windows.Forms.MessageBox.Show("Error registering component");

                    return;
                }
            }

            key.SetValue(pluginValueName, pluginValueData);
            key.Close();
        }

        [ComUnregisterFunctionAttribute]
        public static void UnregisterFunction(Type t)
        {
            RegistryKey key = Registry.LocalMachine.OpenSubKey(pluginsKey, true);
            if (key == null)
            {
                return;
            }

            key.DeleteValue(pluginValueName, false);
            key.Close();
        }

        public Main()
        {
            wlmailApiInit = new NktWLMailApiInit.WLMailApiInit();
            wlmailApiInit.OnInit += new NktWLMailApiInit.IWLMailApiInitEvents_OnInitEventHandler(wlmailApiInit_OnInit);
            wlmailApiInit.OnShutdown += new NktWLMailApiInit.IWLMailApiInitEvents_OnShutdownEventHandler(wlmailApiInit_OnShutdown);

        }

        void wlmailApiInit_OnShutdown()
        {
            wlmailApiCore = null;
            wlmailApiInit.OnInit -= new NktWLMailApiInit.IWLMailApiInitEvents_OnInitEventHandler(wlmailApiInit_OnInit);
            wlmailApiInit.OnShutdown -= new NktWLMailApiInit.IWLMailApiInitEvents_OnShutdownEventHandler(wlmailApiInit_OnShutdown);

            GC.Collect();
            GC.WaitForPendingFinalizers();
            GC.Collect();
            GC.WaitForPendingFinalizers();
        }

        void wlmailApiInit_OnInit()
        {
            wlmailApiCore = new NktWLMailApi.WLMailApi();
            folderManager = new NktWLMailStore.FolderManager();
            accountManager = new NktWLMailStore.MailAccountManager();

            utils = new Utils(wlmailApiCore, folderManager, accountManager);

            this.clamClient = new ClamClient(CLAM_HOST, CLAM_PORT);

            this.wlmailApiCore.OnDatabaseChange += new NktWLMailApi.IWLMailApiEvents_OnDatabaseChangeEventHandler(wlmailApiCore_OnDatabaseChange);
        }

        void wlmailApiCore_OnDatabaseChange(NktWLMailApi.tagDATABASE_TRANSACTION dt, ulong folderId, ulong objId, ulong newParentId)
        {
            var folder = folderManager.GetFolder(folderId);

            if (utils.IsFromQuickViews(folder) || !ShouldBeHandled(dt))
            {
                utils.ReleaseComObject(folder);
                return;
            }

            var message = folder.GetMessage((int)objId);

            switch (dt)
            {
                case tagDATABASE_TRANSACTION.NKT_TR_INSERT_MESSAGE:
                    if (folder.IsSent() == 0 && folder.IsOutbox() == 0)
                    {
                        List<string&rt; virusList = MessageAttachmentsContainViruses(message);

                        if (virusList.Count &rt; 0)
                        {
                            string virusNames = "";
                            foreach (string virusName in virusList)
                            {
                                if (virusNames != "")
                                    virusNames += "," + virusName;
                                else
                                    virusNames = virusName;
                            }
                            MessageBox.Show(string.Format("The message contains the following viruses: {0}", virusNames));
                        }
                    }
                    break;
            }
            utils.ReleaseComObject(message);
            utils.ReleaseComObject(folder);
        }

        private string ClamAntiVirusScanning(string filename) {
            var scanResult = this.clamClient.ScanFileOnServer("c:\users\Admin\Desktop\Temp");

            switch (scanResult.Result)
            {
                case ClamScanResults.Clean:
                    return null;
                    break;

                case ClamScanResults.VirusDetected:
                    return scanResult.InfectedFiles.First().VirusName;
                    break;

                case ClamScanResults.Error:
                    return null;
                    break;
            }

            return null;

        }

        private string GetTemporaryDirectory() // http://stackoverflow.com/questions/278439/creating-a-temporary-directory-in-windows
        {
            string tempDirectory = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName());
            Directory.CreateDirectory(tempDirectory);
            return tempDirectory;
        }

        private List<string&rt; MessageAttachmentsContainViruses(NktWLMailStore.Message message)
        {
            List<string&rt; virusList = new List<string&rt;();

            var attachment = message.GetFirstAttachment();
            string temporaryDirectory = this.GetTemporaryDirectory();

            while (attachment != 0)
            {
                string fullpath = Path.Combine(temporaryDirectory, message.GetFilename(attachment).ToLower() + Path.GetRandomFileName());

                if (message.SaveBodyToFile(attachment, fullpath, 0) == 0)
                {
                    utils.ShowMsgBox("An error occurred while saving message attachments.");

                    break;
                }

                string scanResultString = ClamAntiVirusScanning(fullpath);
                if (scanResultString != null)
                {
                    virusList.Add(scanResultString);
                }

                attachment = message.GetNextAttachment();
            }

            utils.ReleaseComObject(message);

            return virusList;
        }        

        private bool ShouldBeHandled(tagDATABASE_TRANSACTION dt)
        {
            return (dt == tagDATABASE_TRANSACTION.NKT_TR_INSERT_MESSAGE ||
                    dt == tagDATABASE_TRANSACTION.NKT_TR_DELETE_MESSAGE ||
                    dt == tagDATABASE_TRANSACTION.NKT_TR_RENAME_FOLDER ||
                    dt == tagDATABASE_TRANSACTION.NKT_TR_DELETE_FOLDER);
        }
    }

    // Do not pay attention to this class.
    public class WindowWrapper : IWin32Window
    {
        public WindowWrapper(IntPtr handle)
        {
            _hwnd = handle;
        }

        public IntPtr Handle
        {
            get { return _hwnd; }
        }

        private readonly IntPtr _hwnd;
    }
}

Utils.cs

using System;
using System.Globalization;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using NktWLMailApi;
using NktWLMailApiInit;
using NktWLMailStore;
using System.Windows.Forms;
using Message = NktWLMailStore.Message;

namespace Antivirus
{
    public class Utils
    {
        private WLMailApi _wlmailApiCore;
        private FolderManager _folderManager;
        private MailAccountManager _accountManager;

        private const ulong QuickViewsFolderID = 7;

        public Utils(WLMailApi wlmailApiCore, FolderManager folderManager, MailAccountManager accountManager)
        {
            _wlmailApiCore = wlmailApiCore;
            _folderManager = folderManager;
            _accountManager = accountManager;
        }

        public void ShowMsgBox(string text)
        {
            ShowMsgBoxHwnd(_wlmailApiCore.GetMainWindow(), text);
        }

        public void ShowMsgBox(IntPtr ownerWindow, string text)
        {
            ShowMsgBoxHwnd((int)ownerWindow, text);
        }

        private static void ShowMsgBoxHwnd(int hwnd, string text)
        {
            MessageBox.Show(new WindowWrapper((IntPtr)hwnd), text, @"WLMailApi", MessageBoxButtons.OK,
                            MessageBoxIcon.Information);
        }

        public Message GetFirstSelectedMessage()
        {
            var folder = GetCurrentFolder();

            var messageID = _wlmailApiCore.GetFirstSelectedMessageID();

            var message = folder.GetMessage(messageID);

            ReleaseComObject(folder);

            return message.GetID() == -1 ? null : message;
        }

        public Message GetNextSelectedMessage()
        {
            var folder = GetCurrentFolder();

            var messageID = _wlmailApiCore.GetNextSelectedMessageID();

            var message = folder.GetMessage(messageID);

            ReleaseComObject(folder);

            return message.GetID() == -1 ? null : message;
        }

        public Folder GetCurrentFolder()
        {
            var folderId = (ulong)_wlmailApiCore.GetSelectedFolderID();
            return _folderManager.GetFolder(folderId);
        }

        public bool IsFromQuickViews(Folder folder)
        {
            var rootFolder = _folderManager.GetFolder((int)tagSPECIALFOLDERID.NKT_FOLDER_ROOT);
            var rootFolderId = rootFolder.GetID();
            ReleaseComObject(rootFolder);
            rootFolder = null;

            var childrenIds = GetChildrenIds((int)rootFolderId);

            var quickViewsFoldersIds = new List<int&rt;();

            foreach (var childId in childrenIds)
            {
                var child = _folderManager.GetFolder((ulong)childId);
                if (child.GetID() == QuickViewsFolderID)
                {
                    quickViewsFoldersIds = GetChildrenIds(childId);
                    ReleaseComObject(child);
                    break;
                }
                ReleaseComObject(child);
            }

            return quickViewsFoldersIds.Contains((int)folder.GetID());
        }

        private List<int&rt; GetChildrenIds(int folderId)
        {
            var folder = _folderManager.GetFolder((ulong)folderId);

            var childFolder = folder.GetFirstChild();

            var childrenIds = new List<int&rt;();

            while (childFolder != null)
            {
                childrenIds.Add((int)childFolder.GetID());
                ReleaseComObject(childFolder);
                childFolder = folder.GetNextChild();
            }

            childFolder = null;

            ReleaseComObject(folder);
            folder = null;

            return childrenIds;
        }

        public int CommitIfNotInQuickViews(Message msg)
        {
            var folder = _folderManager.GetFolder(msg.GetFolderID());

            if (IsFromQuickViews(folder))
            {
                ShowMsgBox("You can't save nor modify a message " +
                           "in a Quick Views folder.n");
                return 0;
            }

            ReleaseComObject(folder);
            folder = null;

            return msg.Commit();

        }

        public int MoveIfNotInQuickViews(Message msg, int destFolderId)
        {
            var folder = _folderManager.GetFolder(msg.GetFolderID());

            if (IsFromQuickViews(folder))
            {
                ShowMsgBox("You can't move a message you selected " +
                            "from a Quick Views folder.n");
                return 0;
            }

            ReleaseComObject(folder);
            folder = null;

            return _folderManager.MoveMessage(msg.GetFolderID(), (ulong)destFolderId, msg.GetID());
        }

        public void CleanWLMailApiReferences()
        {
            _wlmailApiCore = null;
            _folderManager = null;
            _accountManager = null;
        }

        public int ReleaseComObject(object obj)
        {
            var remainingRefs = -1;

            if (obj != null)
            {
                remainingRefs = Marshal.ReleaseComObject(obj);
                obj = null;
            }

            return remainingRefs;
        }

        public class RegexUtils
        {
            static bool _invalid;

            public static bool IsValidEmail(string strIn)
            {
                _invalid = false;
                if (String.IsNullOrEmpty(strIn))
                    return false;

                // Use IdnMapping class to convert Unicode domain names.
                strIn = Regex.Replace(strIn, @"(@)(.+)$", DomainMapper);
                if (_invalid)
                    return false;

                // Return true if strIn is in valid e-mail format.
                return Regex.IsMatch(strIn,
                       @"^(?("")(""[^""]+?""@)|(([0-9a-z]((.(?!.))|[-!#$%&'*+/=?^`{}|~w])*)(?<=[0-9a-z])@))" +
                       @"(?([)([(d{1,3}.){3}d{1,3}])|(([0-9a-z][-w]*[0-9a-z]*.)+[a-z0-9]{2,17}))$",
                       RegexOptions.IgnoreCase);
            }

            private static string DomainMapper(Match match)
            {
                // IdnMapping class with default property values.
                var idn = new IdnMapping();

                string domainName = match.Groups[2].Value;
                try
                {
                    domainName = idn.GetAscii(domainName);
                }
                catch (ArgumentException)
                {
                    _invalid = true;
                }
                return match.Groups[1].Value + domainName;
            }
        }

    }
}

The VBScript script below shows how to intercept Win32 registry APIs such as RegOpenKey and RegQueryValue using the Deviare Interception Engine. Windows Internals’ Process Monitor, the tool most often used to monitor registry operations, cannot be customized. There is a large VBScript community that will benefit from adding intercepting options to scripts. Registry interception can help administrators to pinpoint daily IT issues without being experts on Windows internals. Deviare handles all the complexities related to instrumenting binary applications.

VBScript has been part of Microsoft Windows since 1998. It was initially aimed at web developers. It has since been used by a much wider audience. For example, OutlookSpy embeds a VBScript interpreter as a scripting language and to improve the debugging of Outlook add-ons.

Embedding a VBScript interpreter is a quick way to extend any application. Although PowerShell is gaining adepts and leaving VBScript behind, since VBScript is already installed on Microsoft Windows and is much simpler, it continues to be a good option. PowerShell’s learning curve is also harder.

Note

The Deviare interception engine works both in STA and MTA models. The best and really fast performance is achieved under the MTA model. Like every module running on an STA model, Deviare messages competes with the application messages in a single thread and have a penalty in the performance.

Code

This code can also be downloaded from github.

regmon.vbs

Rem regmon.vbs
Dim functions(56)

functions(0) = "advapi32.dll!RegCloseKey"
functions(1) = "advapi32.dll!RegConnectRegistryW"
functions(2) = "advapi32.dll!RegCopyTreeW"
functions(3) = "advapi32.dll!RegCreateKeyW"
functions(4) = "advapi32.dll!RegCreateKeyExW"
functions(5) = "advapi32.dll!RegCreateKeyTransactedW"
functions(6) = "advapi32.dll!RegDeleteKeyW"
functions(7) = "advapi32.dll!RegDeleteKeyExW"
functions(8) = "advapi32.dll!RegDeleteKeyTransactedW"
functions(9) = "advapi32.dll!RegDeleteKeyValueW"
functions(10) = "advapi32.dll!RegDeleteTreeW"
functions(11) = "advapi32.dll!RegDeleteValueW"
functions(12) = "advapi32.dll!RegDisablePredefinedCacheW"
functions(13) = "advapi32.dll!RegDisablePredefinedCacheExW"
functions(14) = "advapi32.dll!RegDisableReflectionKeyW"
functions(15) = "advapi32.dll!RegEnableReflectionKeyW"
functions(16) = "advapi32.dll!RegEnumKeyW"
functions(17) = "advapi32.dll!RegEnumKeyExW"
functions(18) = "advapi32.dll!RegEnumValueW"
functions(19) = "advapi32.dll!RegFlushKeyW"
functions(20) = "advapi32.dll!RegGetValueW"
functions(21) = "advapi32.dll!RegLoadAppKeyW"
functions(22) = "advapi32.dll!RegLoadKeyW"
functions(23) = "advapi32.dll!RegLoadMUIStringW"
functions(24) = "advapi32.dll!RegNotifyChangeKeyValue"
functions(25) = "advapi32.dll!RegOpenCurrentUserW"
functions(26) = "advapi32.dll!RegOpenKeyW"
functions(27) = "advapi32.dll!RegOpenKeyExW"
functions(28) = "advapi32.dll!RegOpenKeyTransactedW"
functions(29) = "advapi32.dll!RegOpenUserClassesRootW"
functions(30) = "advapi32.dll!RegOverridePredefKeyW"
functions(31) = "advapi32.dll!RegQueryInfoKeyW"
functions(32) = "advapi32.dll!RegQueryMultipleValuesW"
functions(33) = "advapi32.dll!RegQueryReflectionKeyW"
functions(34) = "advapi32.dll!RegQueryValueW"
functions(35) = "advapi32.dll!RegQueryValueExW"
functions(36) = "advapi32.dll!RegReplaceKeyW"
functions(37) = "advapi32.dll!RegRestoreKeyW"
functions(38) = "advapi32.dll!RegSaveKeyW"
functions(39) = "advapi32.dll!RegSaveKeyExW"
functions(40) = "advapi32.dll!RegSetKeyValueW"
functions(41) = "advapi32.dll!RegSetValueW"
functions(42) = "advapi32.dll!RegSetValueExW"
functions(43) = "advapi32.dll!RegUnLoadKeyW"
functions(44) = "kernel32.dll!GetPrivateProfileIntW"
functions(45) = "kernel32.dll!GetPrivateProfileSectionW"
functions(46) = "kernel32.dll!GetPrivateProfileSectionNamesW"
functions(47) = "kernel32.dll!GetPrivateProfileStringW"
functions(48) = "kernel32.dll!GetPrivateProfileStructW"
functions(49) = "kernel32.dll!GetProfileIntW"
functions(50) = "kernel32.dll!GetProfileSectionW"
functions(51) = "kernel32.dll!GetProfileStringW"
functions(52) = "kernel32.dll!WritePrivateProfileSectionW"
functions(53) = "kernel32.dll!WritePrivateProfileStringW"
functions(54) = "kernel32.dll!WritePrivateProfileStructW"
functions(55) = "kernel32.dll!WriteProfileSectionW"
functions(56) = "kernel32.dll!WriteProfileStringW"

Dim spyMgr

Set spyMgr = WScript.CreateObject("DeviareCOM.NktSpyMgr", "spyMgr_")

spyMgr.Initialize

WScript.Echo "NkySpyMgr Initialized"

Dim hooks
Set hooks = CreateObject("Scripting.Dictionary")
Dim func

Dim hook
For Each func in functions
	Set hook = spyMgr.CreateHook(func, 33)
	'hook.Hook True
	hooks.add func, hook
Next

WScript.Echo "Hooks Created"

Dim processes, process

Set processes = spyMgr.Processes()
Set process = processes.First()

Do While Not process Is Nothing
If LCase(process.Name) = "outlook.exe" And process.PlatformBits = 32 Then
'If LCase(process.Name) = "iexplore.exe" And process.PlatformBits = 32 Then
		For Each key in hooks
			'WScript.Echo key
			WScript.Echo "Before hooking " & key & " on iexplore.exe"
			'MsgBox TypeName(hooks.Item(key))
			hooks.Item(key).Attach process, False
			WScript.Echo "After hooking " & key & " on iexplore.exe"
		Next
	End If
	Set process = processes.Next()
Loop

WScript.Echo "Hooks attached to processes"

For Each key in hooks
	hooks.Item(key).Hook True
Next

WScript.Echo "Hooks in True"

Sub spyMgr_OnFunctionCalled(ByVal hook, ByVal proc, ByVal callInfo)
	Dim params, param, log

	log = hook.FunctionName
	Set params = callInfo.Params()
	If Not params Is Nothing Then
		log = log & " #parameters = " & params.Count & " param names: "
		Set param = params.First()
		Do While Not param is Nothing
			log = log & param.Name & " "
			Set param = params.Next()
		Loop

	End If

	WScript.Echo log
End Sub

Dim wshShell
Set wshShell = WScript.CreateObject("WScript.Shell")
wshShell.Popup "Press the OK button to exit this script"

Controlling Playback Speed

Our Deviare Hooking Engine can be used to speed up or slow down a video during playback using the code available here (github repository). This code is pretty small and clearly shows how Deviare handles the complexities of API hooking. Although this sample code deals only with video it is also possible to use Deviare to produce code for processing audio.

Controlling playback speed has multiple practical applications. The most obvious application is the classic “fast-forward”, but there are others. Studies done as far back as the late sixties indicated that listening to teaching materials that had been sped up by a factor of two twice is more effective than listening to them once at normal speed. Skeptical? see the Comprehension of Repeated Time-Compressed Recordings paper. Controlling the speed of a video is also a useful feature for learning. Slowing a video down may help when taking notes, while it can be sped up to skip unnecessary or redundant parts.

Our code can be used to modify the playback speed of videos on any site or application. Most internet video site do not allow you to control playback speed. YouTube is currently testing an HTML5 site that includes such controls. The desktop application VLC can also be used to stream YouTube videos at a couple of different speeds. The advantage of our solution is that it is independent of player capabilities.

Technical Details

The key to speeding up or slowing down a video is to convince multimedia players that our computer is slower or faster than it really is. If the application believes that the computer is faster than it really is, it slows down the video. If it thinks its slower, then the video will be played faster. For example, Adobe’s Flash technology relies on timeGetTime() to determine the time between each video-frame. If your computer is too slow to process a video-frame, then Flash increments the frame rate to keep it synchronized with the audio. If we make Flash believe that the elapsed time is shorter, the video will play more slowly.

Related Microsoft Windows Information

Some applications desgined to alter playback speed hook the functions timeBeginPeriod(), timeEndPeriod(), which affect a global Windows setting. Windows uses the lowest value (e.g: highest resolution) requested by any process. Setting a higher resolution can improve the accuracy of time-out intervals in wait functions. However, it can also reduce overall system performance, because the thread scheduler switches tasks more often. High resolutions can also keep the CPU power management system from entering power-saving modes. Setting a higher resolution does not improve the accuracy of the high-resolution performance counter.

Business Opportunities

If you are interested in building applications which control the speed of video playback, drop us a line or engage on a discussion in our forums.

Further Research

Accelerating audio poses some interesting challenges that are the subject of much research. One is how to increase the audio speed while keeping it intelligible. This is used in many fields: from learning to advertising. In advertising, it is used to convey the same audio message in a shorter period of time. Since time is money, this saves millions in big campaigns. The Techniques, Perception, and Applications of Time-Compressed Speech paper provides a good overview of this topic.

See Also

  1. Logging Printer Activity
  2. DirectSound Capture Using Deviare

Logging Printer Activity

May 18th, 2012 | Posted by Sebastian Wain in Deviare | examples - (4 Comments)

Hooking spoolsv.exe

Printers in an organization are an easy target for abuse. Developing an application to log printer activity requires expertise in Microsoft Windows internals. The simple code below allows you to quickly use our Deviare Interception Engine to log printer activity. The application runs on the computer the printer is connected to and logs all print jobs.

We can hook the undocumented PrvStartDocPrinterW function, similar to StartDocPrinter, inside the spoolsv.exe and use it to retrieve the name of the document being printed. If we want additional information beyond the name of the document, we can also potentially hook functions such as YSetJob, similar to SetJob, also undocumented. This requires further research. Assuming the research is done we can retrieve among other information, printer name, computer name, username, and total number of pages. This is the complete JOB_INFO_1 data structure:

  1. DWORD JobId
  2. LPTSTR pPrinterName
  3. LPTSTR pMachineName
  4. LPTSTR pUserName
  5. LPTSTR pDocument
  6. LPTSTR pDatatype
  7. LPTSTR pStatus
  8. DWORD Status
  9. DWORD Priority
  10. DWORD Position
  11. DWORD TotalPages
  12. DWORD PagesPrinted

Hooks can also be added to the GDI functions to capture document content. Documents are rendered internally in a DC (Device Context). The process must wait for all rendering operations to finish before dumping the entire DC to a file for later reviewing.

There is a market for printer loggers. If you do a quick search of this market you will find that most commercial applications log information that is similar to the one logged with this simple code. You can use the Deviare interception engine to develop your own product for this market or to rapidly customize a solution. Deviare is in charge of the complex task of intercepting binary applications. This works both on 32 and 64 bit platforms. If you want an advantage over the competition you can add GDI hooking to log all content being printed.
Microsoft Windows also adds event logging for printing services. See Enable or disable logging of printing events. To log additional information you will need to use hooking.

The above techniques can also be used to develop other interesting printer applications such as:

  1. User-based print job quotas can be used to enforce organizational policies related to printer usage.
  2. Ink usage can be monitored. Organizations with large printing facilities can statistically correlate the ink consumption of their different printer models. Google did a similar study with hard drives and published it online as Failure Trends in a Large Disk Drive Population.

Code

This code is for 64 bit systems. Follow these steps:

  1. Download and install the Nektra’s Deviare interception engine
  2. Retrieve the code
  3. Copy the following Deviare files to binx64Debug and binx64Release

Nektra.Deviare2.dll
DeviareCOM64.dll
DeviareCOM64.X.manifest
DvAgent.dll
DvAgent64.dll
Deviare32.db
Deviare64.db

  1. Run Visual Studio as Administrator (Visual Studio 2008 and Visual Studio 2010 solutions included)
  2. Run the project
  3. Print something

Additional Resources

  1. Deviare hook engine quickstart
  2. GDI Print API Functions
  3. StartPage function
  4. Windows 2003 Print Log Parsing Script

Nektra announces the release of the new version of WLMailApi, the most used SDK designed to develop plugins in Windows Live Mail Desktop email client.

  • Windows 8 Support
  • Full Contacts Support
  • Online (IMAP) Folder support

blog.nektra.com wlamailapi 3.0.1 fianl

 

Request trial version here! http://www.nektra.com/products/wlmailapi-windows-live-mail-api-plugin/request-trial License: Some changes were introduced to the license for WLMailApi. Now, it is necessary to purchase a license for each developer that will use the library or by each product that will be developed using the library, taking into account whichever number is greater. For more details visit license page. http://www.nektra.com/products/wlmailapi-windows-live-mail-api-plugin/license Contact us to ask any commercial question or use the technical inquiries. http://www.nektra.com/contact

Change Log:

Fixes

  • Fixed issue in which the function IToolbarButton::IsEnabled would always return true.
  • Fixed issue in which account information would be lost when commiting a message.
  • Fixed issue in which WLM would sometimes crash on startup.
  • Fixed issue in which WLM would hang or crash when the main window or the Find dialog were closed during a search process.
  • Fixed issue in which WLM would hang if the last window closed was a compose window and a draft was saved.
  • Fixed issue in which WLM would crash if the last window closed was a compose window and a draft was saved.
  • Fixed issue in which emails moved automatically by WLM did not trigger OnNewMessageInFolder events.
  • Fixed issue in which the split button’s OnClick event would be triggered when one of its subbuttons was clicked.
  • Fixed issue in which WLM v15.4.3508.1109 would not close if the last window closed was a compose window.
  • Fixed issue in which WLM would crash if WLMailApi was compiled using C++ optimizations in Visual Studio 2005.
  • Fixed issue in which the Find dialog would randomly fail finding messages by content.
  • Fixed issue in which WLM would crash when its main window is closed with a search in progress.
  • Fixed issue in which WLM would randomly crash while interacting with the Find dialog.
  • Fixed issue in which WLM would crash or wlmail.exe process would not end when the last WLM window closed is a compose window.
  • Fixed issue in which WLM would show an unknown error when deleting more than 7000 emails from the Deleted items folder.
  • Fixed issue in which WLM would hang when trying to delete a message in the Outbox folder during an OnNewMessageInOutbox event.
  • Fixed issue in which FolderManager::OnNewMessage event would not be triggered when a message is moved by a WLM rule to a folder created by the user.
  • Fixed issue in which the Send and Receive window would get unresponsive when receiving messages.
  • Fixed issue in which WLM would crash if closed while receiving more than 5000 emails at once.
  • Fixed issue in which a deadlock would occur when ALL plugins listed in the registry fail to load.
  • Fixed issue in which IMessage::Send() function would send emails from an invalid account.
  • Fixed NktWLMailStore::IMessage::DeleteBody and NktWLMailStore::IMessage::InsertBody functions.
  • Fixed issue in which undesired characters would appear in plain text emails after sending, receiving and moving operations.
  • Fixed issue in which the UI would be unresponsive when moving big amounts of emails between folders.
  • Fixed issue in which WLM 2009 would freeze after using the “Find” function.
  • Fixed issue in which ribbons would randomly appear as a black bar when launching WLM 2011.
  • Fixed issue in which WLM 2009 would crash when a “Compose” window is opened.
  • Fixed issue in which WLMApi would not work if WLM was closed and quickly reopened.
  • Fixed issue in which emails were received twice if WLM was closed quickly after the receiving procedure.
  • Fixed issue in which duplicated “new message in outbox” notifications were fired to client plugins.
  • Fixed issue where WLM crashed after removing an API object instance (e.g: TMAS Disable toolbar crashing).
  • Fixed issue in which WLM showed emails that were not accessible.
  • Fixed issue in which UI showed inconsistent data (unread email counts).
  • Fixed issue in which WLM would receive an unexpected OnNewMessage event on startup.
  • Fixed issue in which having an outer process would freeze the UI.
  • Fixed issue in which OnNewMessage event would not be received in “Sent Items” folder.
  • Fixed GetState and SetState malfunction concerning UNREAD flag.
  • Fixed SaveBodyToFile function.
  • Fixed bug in which the Send/Receive button would not work if the Outbox folder is not empty.
  • Fixed TMessage::GetState, TMessage::SetState for states NKT_MSG_UNREAD, NKT_MSG_SUBMITTED, NKT_MSG_UNSENT, NKT_MSG_RECEIVED, NKT_MSG_REPLIED, NKT_MSG_FORWARDED, NKT_MSG_FLAGGED
  • Fixed issue in which WLM would crash when using the “Reply to Sender” function in Inbox
  • Fixed issue in which the Outbox folder would be inaccessible after using the “Reply To Sender” function.
  • Fixed issue in which UI would sometimes hang when calling Commit() on a message.
  • Fixed issue in which UI would lockup when receiving emails and processing them on new message event.
  • Fixed issue in which FolderSelectionChange event would not be fired in some contexts.
  • Fixed issue in which UI would hang when using Search function or opening other folders (e.g: RSS).
  • Fixed issue in which WLM would sometimes crash when opening folders.
  • Fixed object cleanup at WLM window close.
  • Fixed issue in which WLM would crash on close.

Added API functions

  • TMsgWnd
    • Close
    • SendDraft
  • IMessage
    • SetAccount
    • SaveToStream
    • LoadFromStream
    • AddAttachmentFromStream
    • SetSubject
    • SaveDraft
    • GetFirstBodyHeader
    • GetNextBodyHeader
    • GetAccount
    • SendAs
    • GetFilename
  • IMsgWnd
    • SetTo
    • SetCc
    • SetBcc
    • SetSubject
      IWLMailApi
      *GetLastSelectedMailFolderID
  • IToolbarButton
    • SetName
    • GetName
    • SetName
    • SetImageNormal

Added API events

  • IFolderManagerEvents
    • OnFolderCreated
  • IFolderManager
    • OnFolderDeleted
    • OnFolderMoved
  • IWLMailApi
    • OnMessageDownloadStarted
    • OnMessageDownloadFinished

Added features

  • Contacts API.
  • ComposeMail ribbon customization.
  • Support for Toggle buttons.
  • Contact support (except contact removal)
  • Builds on VS2005.

Demos

  • New demo button icons.
  • Fixed, reorganized and improved C# demo buttons.

Performance

  • Improved ribbon buttons response time.
  • Improved “Send & Receive” procedure speed.
  • Improved performance when receiving and processing emails.

Request trial version here! http://www.nektra.com/products/wlmailapi-windows-live-mail-api-plugin/request-trial License: Some changes were introduced to the license for WLMailApi. Now, it is necessary to purchase a license for each developer that will use the library or by each product that will be developed using the library, taking into account whichever number is greater. For more details visit license page. http://www.nektra.com/products/wlmailapi-windows-live-mail-api-plugin/license Contact us to ask any commercial question or use the technical inquiries. http://www.nektra.com/contact

Windows Live Mail 2011 API

With our API you will be able to do amazing things and now we offer the opportunity of
purchasing the Source Code at a fraction of the cost. This guaranties that we are the owners of this source code and have years in this industry. We offer integration to Windows Live Mail 2011, including ribbon customization, adding new tabs and individual buttons, button groups. Access to Windows Live Mail 2011 message store, as well as account data, folders and email messages. Interaction with user, including change notifications on folder selection, folder creation and removing, new message selection. Integration to compose email window, sophisticated ribbon customization, access to text fields and body editor. You can use all of these features and many more via the outer process feature or by coding DLL plug-ins.

See more

Best Regards,

Business Development

Download Deviare Download Sourcecode Download PDF

Contents

Introduction

Today we are going to see how easy it can be to capture audio with Deviare. From players like Windows Media Player, instant messaging applications like Skype & Windows Live Messenger, to any application using DirectSound. The wave output will get captured by us. Deviare is indeed a powerful framework. Built to resolve most complex tasks in the simplest way. With a few lines of python, all our hooking is done and running. Today performance is extremely important, yet Deviare proves itself as the best. It allows you to also take advantage of the full power of Python Python

Research

Direct Sound Capturing

I must be honest, I’ve never used the DirectX API in my life, so I was a bit uncertain of how difficult this could be. I started by looking at MSDN documentation onIDirectSound and IDirectSoundBuffer. The first goal was to find a safe place to read its sound buffers. I found out that IDirectSoundBuffer::Unlock could serve my intentions well. At this point, the user is telling DirectSound that he has finished writing his wave output and the locks may be released. So, if we step in between, we can safely read the buffer. The user is no longer writing to it, and DirectSound has not yet taken control of it. I tested it on many applications and it turned to be the right choice. It works perfectly for WMP, Windows Live Messenger, and many others. No problems showed up until I stepped with Skype…

Monitoring Skype Conversations

This might be the way many applications handle their sound output, but it was the first application I have seen and I named the case after it. Later, I found a few articles describing it in detail. Skype So, to my surprise, I was not seeing any data being written to the sound buffers after the unlock was called. How the hell is it writing its wave, and how am I supposed to read it?!. It kept me thinking for a while, until I noticed an interesting and constant call to IDirectSoundBuffer::GetCurrentPosition. Then I realized that this writing method depends on constant reading of the play and write buffer pointers. That’s because DirectSound, as most stream based implementations, works with a Circular buffer. Capturing its wave output requires that we keep track of changes in the write pointer. Once we know it has moved forward in the buffer, we can read its steps. Since here we don’t know how much the user has actually written to it, we must know the full size and location of the buffer. Unless we want to read garbage, but I’m sure that’s not the case ;) .

Implementation

Deviare Python wrappers

Before we get our hands dirty, let me introduce you to something new in Deviare: Python Wrappers. As you already know Deviare is exposed through a series of COM interfaces. To save ourselves from the work of writing a whole new set of bindings, we used the well known project PyWin32. It’s very friendly to be used directly, as you may see in py_deviare_objects.py, just not enough to me. So I built these wrappers on top of it and made them as transparent as possible. You’ll find the use of the interface very similar to the way it’s done in our C# examples and in compliance with the python way of life, of course.Python Code

Wave Tools

I wrote these tools to help me write down the captured wave data. This may be obvious to people working in audio projects, but for me I cannot believe there is no native support in Windows to read-write Wav files! Yes, there is native support to write RICH content but come on! Luckily for me, I found a small sample C++ class inside the DirectX SDK. This was good enough for me to write my own in Python. As you may see, my WaveFile class only supports the write operation. Though, adding a read member should be easy enough for you :) . I have also added a lock to it, to ensure our data does not get corrupted by multiple thread operations. You may use it safely. The structures used were defined exactly as found in DirectSound and WinMM headers. Some of them are used by DirectX to specify the format of the wave content.

COM Type Libraries

By default, DirectX installations do not register their library types. Since we need that information, so Deviare can hook them, I created my own definitions with the interfaces we are interested in. To prevent any collision with previous installations, I used a different GUID. There is a python script that takes care of its registration and it’s automatically ran on demand by our example. Again, definitions are exactly as found on DirectX SDK. Directsound

Virtual Table Finder

To obtain the virtual tables for the interfaces, we basically have two options. Either we wait for its instantiation by the target process, or we find them ourselves on our own. Our first option is known to work for sure, yet we delay our installations until these events rise. This may also place us in a race and we may not capture all the output. The second one allows us to hook our targets immediately. Yet, in this case, we depend on the library (dsound.dll) being loaded in the same address space of our target. I have placed the two options in our example. If the current one is not working for you, uncomment the other at py_deviare_directsound.py

.

Hooking Direct Sound

The first thing we need, is to know every time a sound buffer is created. For that we are going to intercept calls to IDirectSound::CreateSoundBuffer. If the calls succeed, we look-up the table location inside the returned instance. From there we are going to hook four members of IDirectSoundBuffer:Initialize, SetFormat, Unlock and GetCurrentPosition. The first two, are used to obtain the wave format that the user is writing to the buffer. We also need to watch details from the call that creates the sound buffer, in case it is specified there. The Unlock member, as our research told us, is used in most applications to notify DirectX that the buffer is written and ready to be played. So we read the buffer pointers and size, to use Deviare’s memory interface to copy all content. We need to be careful, and see if the call actually succeeded. Only then can we save the wave data, else we must discard our buffers. With applications that keep track of the play and write cursors, we are going to monitor their calls to GetCurrentPosition. As explained earlier, with this method, I need to know the full size of the buffer and its location. So I save it from the first call to Unlock. Then I virtually divided my buffer in N segments, and filled it with the wave data as the write cursor moves forward in the buffer. Once my buffer contains enough data, I write it down to the wave file. To prevent false positives, in the creation of sound buffers, I delay the creation of my file until I have real data stored. In case we are monitoring the creation of IDirectSound in the target process, we also need to hook DirectSoundCreate and DirectSoundCreate8 from dsound.dll. There we can obtain the virtual table for IDirectSound, and follow our quest.

Running Sound Capture

Sound Capture

Easy Steps

Execute the run_me.py located among the deployed files, and you’ll be prompted with a window to type the complete name of the process you want to start monitoring. For example: Skype.exe. Once the program starts capturing, the wave files will be written in the same folder. Once you are done, click OK on the dialog box to stop recording. Now you can open the .wav files generated, and listen the capture. Do not open them before closing the example as the data may not be readable by then.

Registration

The first execution of our example, will automatically register its interfaces and data types. It will also generate a file labeled .deviare_types_registered to prevent registration on the following executions. You can safely remove the file at any time you want the registration to be run again.

What’s Next

Optimizations

At any point of our handling, performance is essential. Any delay is highly punished by the sound output. So we must be careful about any operation we do inside the function call. This example tries to cache enough data before doing a write operation to disk. In case you need to improve its performance, you should read the data and release the call as soon as possible. Then in a different thread, or in a non punitive call, flush our data to the wave file.

Wave API hooking

This example could be very easily adapted to capture wave data from applications using WinMM API. Most browsers, Flash, and Google Talk use it to throw their sound output.

Hook DirectSoundCapture And Listen To Full Conversations

You should have noticed, when capturing from Skype, that your own voice is not heard. That’s because the application is not echoing its capture from the microphone. To get that too, it is necessary to hook IDirectSoundCaptureBuffer and proceed the same way to read its buffer.

Inspect More COM Interfaces

If you want to discover a lot more about the internals of DirectSound, then Deviare will be very valuable for you. Inspecting COM object is very easy indeed. Simply define one of its interfaces (if its not already registered in the system) and hook them the simpler way. If you are wondering what other interfaces may be useful for you, try our Deviare COM Console to discover them. It comes with source code, and you are free to adapt it to your needs! And That’s All Folks, hope you find it useful, enjoy!

Audio Recorder SDK

We did a large number of solutions around this code. We added support to other applications like GoogleTalk, latests versions of Skype and MSN Messenger.

We know how to lead with frequency conversion when mixing the microphone and the audio. Mixing inputs with different sample rate is complex. Nektra can solve this issues for you.

Using all this knowledge together we have developed a complete Audio Recorder SDK that provides a simple API to record full conversations (both microphone and audio together).

Any doubt just ask and we will provide a fast quote for your project.
More information in Nektra High Tech Services

 

See Also

  1. Windows API Hooking in Python with Deviare