TraceListeners, Data Bindings, Styles etc

Its been a while since I've posted and this is more of a link dump than anything else to aid other people debugging their applications.  WPF is a complicated beast and sometimes debugging (bindings, styles, etc) can be a tiresome task.  Here are a bunch of links to useful information about the web:
Dependency Properties might be set in multiple places - this is an article on MSDN describing precedence in setting dependency properties in styles or XAML.  The main take-home thing to remember here is that styles do not override values set for dependency properties at the XAML level.  You'll see that XAML comes in at #3, style triggers next, then style setters.

Secondly, if you're using a data template, applying styles via keys in a resource dictionary is probably a safer bet than assuming that key's in the resource dictionary for that TargetType will be respected.

Finally, check out this article from the WPF Disciples website about debugging using trace listeners.  This works in SharpDevelop too, you'll need to add an application config if it does not exist via the Misc > app.config option, or add existing to the project.

Also, for a great explanation of the finer points of MeasureOverride and ArrangeOverride etc, see Dr WPF's great article ItemsControl: 'P' Is For Panel, part of his excellent A-Z series on WPF.  I highly recommend going through all of the posts in this series.

Shell/Internet Explorer Automation in C#

It seems to me one of the great travesties of programming that a language with the power of C#, with all its LINQ goodness, nullable types and features such as the yield keyword, has such poor support for the COM architecture as compared to say, Visual Basic or Powershell. This lack of support is, of course to some extent by design: Visual Basic developed alongside the COM architecture while C# is relatively speaking a newcomer to the scene. Powershell of course manages its rich support of COM objects thanks to deep voodoo such as the use of IUnknown, type libraries, and so on. But what about C Sharp?

One of the recommended ways of COM interop in C# development involves using Interop assemblies generated by programs such as tlbimp or AxImp. However, for people working on a restricted environment where such tools are not available, no way exists to translate the functions the COM interface provides into .NET classes. So what to do then?

The incomplete answer, of course, is: use reflection. Our intent being to provide information to the development community at large, it is worthwhile providing a little insight into how this can be done.

The first thing to do is to create the COM object.  Because we will manipulating this via reflection, we start by assigning it to a simple System.Object, like this:

object ShellApp=Activator.CreateInstance(Type.GetTypeFromProgID("Shell.Application"));

Activator is in the System namespace, like Type.  If we look at the definition for CreateInstance we see the Type definition returned from GetTypeFromProgID contains the information required to instantiate the COM object.  This is essentially the C# equivalent of VB's CreateObject functionality.  From here, reflection is used to access the methods.

It is useful to use Powershell in exposing the COM interfaces, and COM members can be found via the Get-Member cmdlet.  But do not rely on it for everything: for instance, the FireEvent method on several classes (such as mshtml.HTMLSelectElementClass, etc), should actually be called as fireEvent (with a lower-case f). However, Powershell exposes the member as FireEvent.  Attempting to call this with reflection results in a name not found exception.  So when in doubt, check the relevant msdn documentation for the interface in question.

With that out of the way, lets begin.  The majority of your calls to the COM interface will look like one of the following functions (which I recommend you re-use and bundle within a class if you're going to be using them constantly):

        object GetCOMProp(Object sourceObj,String propName,object[] Param) {
            try {
                return sourceObj.GetType().InvokeMember(propName,BindingFlags.GetProperty,null,sourceObj,Param);
            } catch (Exception e) {
                Console.WriteLine("Could not get property " + propName + " on " + sourceObj.GetType().FullName + ": " + e.ToString());
                return null;
            }
        }

        object SetCOMProp(Object sourceObj,String propName,object[] Param) {
            return sourceObj.GetType().InvokeMember(propName,BindingFlags.SetProperty,null,sourceObj,Param);
        }

        object CallCOMMethod(Object sourceObj,String methodName,object[] Param) {
            return sourceObj.GetType().InvokeMember(methodName,BindingFlags.InvokeMethod,null,sourceObj,Param);
        }

You'll notice these methods are very similar.  Firstly, they call GetType() on the object  to obtain type information.  This could be a COM object, or a CLR class exposed via COM, or something else.  Then, InvokeMember is used - we do not use GetProperty or GetMethod or any other such methods for COM, since usually your object type will be System.__ComObject - a type which has no method or property information exposed on a per COM basis.

The method or property name is usually case sensitive.  To return a property, the binding flags are set to GetProperty; to set a property, SetProperty; to call a Member which has parameters, use InvokeMember.

Once you have your object returned, you need to cast it to whatever type you require.  Often, this will be another object, in order to call a subproperty.  However it can also be a String, or some other value.

If you are expecting a function to return a non nullable type such as int or bool, you should cast to the related nullable type, ie bool? or int?.  This is because it is possible for InvokeMember to return null for various reasons.

Now that you have this out of the way, your implementation becomes a simple matter of invoking methods within try blocks, checking for nulls, and utilizing the very ample API documentation available both online and through the use of Powershell.

Here is an example function:

        public void ClickElementByName(object myWin,String elementName) {
            if (myWin == null) return;
        
            object[] noparm=new object[] {};
            object myDoc=GetCOMProp(myWin,"Document",noparm);
            object searchObj;
        
            if (myDoc == null) {
                Console.WriteLine("ClickElementByName: myDoc is null");
            } else {
                searchObj=CallCOMMethod(myDoc,"getElementsByName",new object[] {elementName});
                if (searchObj != null) {
                    searchObj=CallCOMMethod(searchObj,"namedItem",new object[] {elementName});
                    if (searchObj != null) {
                        CallCOMMethod(searchObj,"fireEvent",new object[] {"onclick"});
                    }
                }
            }
        }

Lets take a look at how this works.  It receives an object of a window you have already identified, perhaps by iterating the Items method on the Windows() IDispatch of a Shell.Application object.  You can identify the interfaces called in Powershell by creating such an object and then assigning the return value of each function to a variable, and running ",$var|gm" (note the comma, which prevents Powershell from iterating any interface that the object may provide).  As you will see by following the above instructions, all we are doing is ensuring that we call the appropriate interfaces to iterate down the COM object's properties and methods chains.

Note that you can also work with such COM objects using the new .NET 4.0 dynamic variable type, but I'm sure you already have found that online.  When you're looking for backward compatibility and don't want to rely on interop libraries, Reflection is a satisfying, if cumbersome solution.

CollectionViewSource and auto-completion

Jason Kemp has a good article on using the CollectionViewSource class of WPF to create an autocomplete implementation. The first part of the series is at this url.

This can be implemented in Powershell very easily to provide a means of searching and selecting from ComboBox's whose source is a list of PSObject's. The basic procedure:

• Create a list as a New-Object system.collections.objectmodel.observablecollection[System.Object], and $obj.Add each new-object PSObject passing its -Property hashref.
• Create a New-Object system.windows.data.collectionviewsource and set its source to the ObservableCollection.
• Optionally create a number of New-Object system.componentmodel.sortdescription's and add them to the sortdescription collection of the CollectionViewModel's view.
• Further on in your code, assign the CollectionViewModel's View as the ItemsSource of a ComboBox or other similar class, and add a TextChanged event to the corresponding textbox to refresh the view.
• Finally, add a Filter to the View by casting a scriptblock as a [predicate[object]] with a single parameter which you can cast back to a psobject and perform your match against. You can find the related textbox by doing a FindName on the textbox's x:Name property, set in your form's XAML.

Validation in Windows Presentation Foundation

Paul Stovell has a great article on the WPF error validation model at http://www.codeproject.com/KB/WPF/wpfvalidation.aspx.

The validation model in WPF is robust and tied in with Bindings, Using it in Powershell would be a matter of:

• Implementing a C#/VB.net class as a wrapper to System.Management.Automation.Scriptblock to override the Validate() method.
• Programmatically set the validationrules property binding of the binding to include an instantiation of the new class.

DynamicResource as a Binding using Styles

Suppose you've implemented an IValueConverter that you want to utilise on a DataTemplate. The problem is, this data template has been defined as a separate XAML template file, external from your VB code, to improve the extensibility of the program into the future. You don't want to have to dynamically generate the xaml to include the namespace reference to your assembly. What then?

The answer is: use WPF styles. This will allow you to programmatically bind parts of an XAML element, with the details of this determined via dynamic resources upon use of the template. This means you can create the binding and specify the IValue converter using code. Here's how you would do it in VB.net:


Dim sActivationStyle as Style=New Style()
Dim activationBinding as Binding=New Binding()
activationBinding.Path=new PropertyPath("activationDate")
activationBinding.Converter=new DateConverter()
sActivationStyle.Setters.Add(new Setter(TextBlock.TextProperty,activationBinding))


Here, DateConverter is your IConverter class you have created to format the DateTime value specified by the activationDate property of the relevant object. Then you use the binding like this:

<TextBlock Style="{DynamicResource activationDate}" DataContext="{Binding}" />

Voila! You now have a datatemplate, loaded at runtime, with a binding set as a dynamic resource.

UIAutomation, Unmanaged APIs and Powershell

Consider this scenario: you have an old, legacy program written in Visual Basic 6 (ThunderRT6 style controls). It is closed source and gets data back from an unknown location to display needed information in a list view format. Unfortunately, the output is somewhat lacking: no column sorting, no copying or parsing of the output, just viewing it.

How do you get this information back directly from the running program, so you can use it for your own needs?

The answer is via the .NET UI Automation interface. The assemblies in question are UIAutomationTypes and UIAutomationClient, among others, which you can add to your program's runspace via Add-Type using the -AssemblyName parameter.

The first thing you need to do is get the desktop:
$desktop=[Windows.Automation.AutomationElement]::RootElement

Once you have the desktop, you can find your program.

$myprogram=@($desktop.FindAll("Subtree",[Windows.Automation.Condition]::TrueCondition)|where-object {$_.Current.Name -match "My Program's Title"})[0]

Once you've verified your program's form is running and able to be found, you can find items within its control space:

$lview=@($myprogram.FindAll("Subtree",[Windows.Automation.Condition]::TrueCondition)|where-object {$_.Current.ClassName -match "ListView20"})[0]

The example above is returning classes matching ListView20. We are looking for the control ListView20WndClass in this case - an old VB6 list view control. Once we have it, $lview.Current.NativeWindowHandle gives us the hWnd to do more technical stuff...


Add-Type -TypeDefinition @"
using System;
using System.Runtime.InteropServices;

namespace Win32 {
public class APIs {

[DllImport("user32.dll", EntryPoint="SendMessage", CharSet=CharSet.Auto)]
public static extern IntPtr SendMessage(IntPtr hWnd, int msg, IntPtr wParam, IntPtr lParam);
[DllImport("kernel32.dll")]
public static extern IntPtr VirtualAllocEx(UInt32 hProcess,IntPtr lpAddress,IntPtr dwSize,UInt32 flAllocationType,UInt32 flProtect);
[DllImport("kernel32.dll")]
public static extern Boolean VirtualFreeEx(UInt32 hProcess,IntPtr lpAddress,IntPtr dwSize,UInt32 dwFreeType);
[DllImport("user32.dll")]
public static extern IntPtr GetWindowThreadProcessId(IntPtr hWnd,out int procId);
[DllImport("kernel32.dll")]
public static extern IntPtr OpenProcess(UInt32 dwDesiredAccess,Boolean bInheritHandle,UInt32 dwProcessId);
[DllImport("kernel32.dll")]
public static extern Boolean WriteProcessMemory(UInt32 hProcess,IntPtr lpBaseAddress,IntPtr lpBuffer,IntPtr nSize,out int lpNumberOfBytesWritten);
[DllImport("kernel32.dll")]
public static extern Boolean ReadProcessMemory(UInt32 hProcess,IntPtr lpBaseAddress,IntPtr lpBuffer,IntPtr nSize,out int lpNumberOfBytesRead);
[DllImport("kernel32.dll")]
public static extern Boolean CloseHandle(UInt32 hObject);
}

[StructLayoutAttribute(LayoutKind.Sequential)]
public struct LVITEM {
public uint mask;
public int iItem;
public int iSubItem;
public uint state;
public uint stateMask;
public IntPtr pszText;
public int cchTextMax;
public int iImage;
public IntPtr lParam;
}

}
"@

$lvpid=[Intptr]::Zero
$null=[Win32.APIs]::GetWindowThreadProcessId($lview.Current.NativeWindowHandle,[ref] $lvpid)


Here we have loaded functions from Kernel32 and User32 that will allow us to accomplish our goals. We get back the process id of the Window handle which allows us to write to the processes memory space.

Why would you want to do that, you ask? Well, since LVM_GETLISTTEXT, the message which allows you to return the text of a listview item, is above 1024 (the value is 0x102D specifically). Below this value (WM_USER), Windows performs automatic memory marshalling for you, however for custom controls such as this listview control, passing it a pointer to memory in Powershell's runspace with its managed code and shiny bells and whistles, would be a very bad thing (tm). Thats because this pointer is specific to Powershell's address space. The same pointer in the programs address space makes no sense and probably would overwrite critical program data.

Not a problem though, the function OpenProcess will allow you to allocate data in the other program's address space and copy data back from this memory.


$lvitem=New-Object Win32.LVITEM;
$lvphandle=[Win32.APIs]::OpenProcess(0x1f0fff,$False,$lvpid);
$lvstructmem=[Win32.APIs]::VirtualAllocEx($lvphandle.ToInt32(),[IntPtr]::Zero,[System.Runtime.InteropServices.Marshal]::SizeOf($lvitem),0x3000,0x04);
$lvstringmem=[Win32.APIs]::VirtualAllocEx($lvphandle.ToInt32(),[IntPtr]::Zero,1024,0x3000,0x04);
$numitems=[Win32.APIs]::SendMessage($lview.Current.NativeWindowHandle,0x1004,[IntPtr]::Zero,[IntPtr]::Zero)


Here we have created an LVITEM structure, described in the MSDN documentation for LVM_GETITEMTEXT, and allocated some memory in the other processes memory space for it and a character buffer large enough to house the string. We then get back the number of items in the list view (LVM_GETITEMCOUNT is 0x1004). This can be done without any marshalling - SendMessage just returns this value as an int.

We would then need to loop over the items in the listview starting from index zero.


$lvitem.pszText=$lvstringmem;
$lvitem.cchTextMax=1024;
$lvitem.iSubItem=0


Here we set the LVITEM structure to reference the memory in the other process. We then need to copy it to a local buffer and copy this buffer ultimately to the processes memory space:


$ptrtolocalbuff=[System.Runtime.InteropServices.Marshal]::AllocHGlobal(1024);
....
[System.Runtime.InteropServices.Marshal]::StructureToPtr($lvitem,$ptrtolocalbuff,$False);
$null=[Win32.APIs]::WriteProcessMemory($lvphandle.ToInt32(),$lvstructmem,$ptrtolocalbuff,[System.Runtime.InteropServices.Marshal]::SizeOf($lvitem),[ref] $outBytes)


Finally, we call SendMessage with the LVM_GETITEMTEXT message to get back the text of the subkey, and copy it back to our own memory space. We can then convert the buffer pointer to a string. In this case the VB6 program was using ANSI string encoding.


$bytesRet=[Win32.APIs]::SendMessage($lview.Current.NativeWindowHandle,0x102D,$lvitemnum,$lvstructmem)
$null=[Win32.APIs]::ReadProcessMemory($lvphandle.ToInt32(),$lvstringmem,$ptrtolocalbuff,1024,[ref] $outBytes);
$uname=[System.Runtime.InteropServices.Marshal]::PtrToStringAnsi($ptrtolocalbuff);


So there you have it. You would need to perform similar marshalling and calling of SendMessage for each field in the listview you wanted to get back as text.

From there you can do what you want with the data. Don't forget at the end however, to deallocate all the allocated memory:


$null=[Win32.Apis]::VirtualFreeEx($lvphandle.ToInt32(), $lvstringmem, 0, 0x8000)
$null=[Win32.Apis]::VirtualFreeEx($lvphandle.ToInt32(), $lvstructmem, 0, 0x8000)
$null=[Win32.APIs]::CloseHandle($lvphandle.ToInt32())
$null=[System.Runtime.InteropServices.Marshal]::FreeHGlobal($ptrtolocalbuff)


System.Net.HttpWebRequest and PowerShell, part 2

In the last post, we covered making a basic web submission and getting a response back using .NET and Powershell.  This is all fine and dandy, but without interpreting this data, you'll find yourself unable to postback to the application to login.

Sadly, .NET has no simple way inbuilt (without third party modules) to parse HTML into a tree structure capable of being easily navigated to extract information.  Again, if you were using InternetExplorer.Application, this provides an interface onto the DOM, but this does not allow you full control of the process of downloading data, which is what we're after here.  Its also extraordinarily slow: using Measure-Object, you can determine that parsing the innerHTML of an element using regular expressions can be from 10-20 times faster than returning  the row and column objects via the API and iterating over their innerText.

So let's assume we're using regular expressions.  How do we go about this?

Consider the following regular expression:
$r=[regex] "<($([String]::join('|',$tagnames)))((.|\n)+?)>"

Given an array $tagnames, we match an element that is any one of these tagnames, followed by the shortest possible number of characters, and then a closing right angled bracket.  This does not match (a) elements that are syntactically incorrect or (b) elements that do not have any attributes.

We can then extract the attributes via the following regular expression:
$r2=[regex] '((\s|\n)+)(\w+)=("[^"]*?"|[^\s]+)';

This extracts from the second match group of the first regular expression, the name and value pairs that constitute attributes.  They can be enclosed in quotes, or be a run of characters not containing a space.  You can abstract this to include the single quote as a valid enclosing character as well, I leave this as an exercise to you.

Lets suppose we want to extract all the input tags (including input type=hidden), filter them, and post some of then back to the application.  Tags such as __VIEWSTATE and other ones beginning with underscores are a good example of this.  They need to be transmitted with the session state cookie (which is not easily extracted from IE itself, but is easily gotten via System.Net.HTTPWebRequest)  in order to retain our login session.   Using these regular expressions and iterating the matches from the second will allow us to extract the name/value/type fields and extract them as required, prior to  posting them back.

So how do we post them back to the form?  Well, a post request needs to send data representing the post request to the server, and the data must be url encoded.

$pdata=[String]::Join('&',@($namevaluepairs|%{"$($_.name)=$([system.web.httputility]::UrlEncode($_.value))";}));

The above translates an array of hashrefs containing name,value pairs, into post data.  Note that you must first:
[System.Reflection.Assembly]::LoadWithPartialName("System.Web");

in order to load the assembly containing the UrlEncode function.

Finally, you must translate the data of the post request to the appropriate character encoding, and post it off.  Here is an example of that translation:

$bytez=[System.Text.Encoding]::UTF8.GetBytes($pdata);

This is translating the string $pdata into a byte array representing the string in the UTF8 character encoding (that most commonly used by web servers).

Finally, you must let the server know how much data you will be sending, so it can allocate memory for your request, as well as telling it what format the data is in:


    $req.ContentType = "application/x-www-form-urlencoded";
    $req.ContentLength = $bytez.Length;
    $reqstream = $req.GetRequestStream();
    $reqstream.Write($bytez, 0, $bytez.Length);
    $reqstream.Close();


This completes your postback to the server and you now read back the data in the usual way (described in part 1).

Now you can check the results of your download and react in your program accordingly, saving the file or textual data to an appropriate location.