Why can't Powershell update objects which contain extended ascii in their names ?

Dennis Hanuska

November 24, 2015 14:49

Answered

A powershell script has been created which uses the Invoke-RestMethod to perform a /Config/FindObjectsOfClass query for certificate objects via the VTPP RestAPI.

The script executes successfully and returns a list of VTPP DNs that represent config certificate objects matching the specified criteria.

The script then steps through each of those returned DNs and attempts to perform a /Config/Write on each DN. However, if any of those DNs contain an extended ascii character (eg. dash x0150) then the VTPP rest api returns a 400 ObjectDoesNotExist error.

The object, does in fact exist though and the user used for RESTAPI authentication does have access to the object.

Date Votes

6 comments

0
Dennis Hanuska November 24, 2015 15:17
The problem ultimately sits with the Invoke-WebRequest method but also impacts any methods which extend this class such as the Invoke-RestMethod class.

Both of these methods convert extended-ascii characters to their asci equivalents before sending them out on the wire. Thus, if we have any objects or attribute names/values in VTPP that contain extended ascii characters then explicitly accessing them via Powershell will fail. The problem is only one way, received data from VTPP is returned correctly from those methods, even if the data contains extended ascii characters, but posting data that contains extended-ascii results in the posted data being converted to standard ascii.

Thus we see the behavior as below:
1. the Invoke-RestMEthod -Uri /Config/FindObjects... successfully returns a DN which contains an extended ascii character [DN=\VED\Policy\Certificates\Cert — Nov 24 (note the long dash x0151, it is not a hyphen but a dash)
2. The returned DN is passed as the ObjectDN parameter into Invoke-RestMethod -Uri /Config/Write...
3. When the VTPP RestAPI receives the request the DN has been changed to DN=\VED\Policy\Certificates\Cert - Nov 24 (note how the dash has been changed to a hyphen x02d) . This change occurred on the client system before the data was received by VTPP (verified with WireShark).
4. VTPP returns an object not found error
VTPP is okay with extended ascii. Everything from the ui to the database to the WEBSDK support it just fine. Interacting with the RESTAPI via executable clients implemented in other languages (eg. c# and RESTSharp) work fine as well. Using the Powershell Invoke-RestMethod is the only place I've seen this so far.

There may be a work around to this which would still allow you to use Invoke-RestMethod but I have not discovered it yet.

There are three ways I can think that we can have objects created in VTPP that contain extended ascii characters in their names.
1. Explicitly entered by the user from the keyboard (takes a bit of effort using a standard keyboard, eg. ALT 0150 on the numeric keypad gives you a dash)
2. Cut and paste from our pdf documentation.
- the original pdf documentation for the client subsystem contained sample macros for client work object definitions
- when cutting and pasting the naming pattern macro from the pdf doc it results in a naming pattern that has an “en-dash” x0150 (looks like a hyphen but it’s not)
- this results in certificate objects being created that contain an “en-dash” in the name
importing / discovering certificates

if a certificate contains an extended asci character in the subject then it is likely we will end up creating a config object that contains extended asci in the object name

Dennis Hanuska November 24, 2015 15:27

I should have mentioned the Worse case scenario which is that we don't receive an "objectNotFound" error but in fact map to a different object.

Walter Goulet November 24, 2015 15:33

This sounds very similar to the issue described in this TechNet article: https://social.technet.microsoft.com/Forums/windows/en-US/d795e7d2-dcf1-4323-8e06-8f06ce31a897/bug-invokerestmethod-and-utf8-data?forum=winserverpowershell . That article implies that the encoding used by the Invoke-RestMethod call uses a default character set for encoding data sent to the webservice. Could you try including the character set encoding explicitly in the 'ContentType' header like this:

Content-Type: text/html; charset=utf-8

So in your PowerShell script, you would invoke the method as follows:

Invoke-RestMethod -ContentType "application/json; charset=utf-8"

to send the data over as unicode. I would also be curious to see the headers sent by the working tools to see if they are setting the Content-Type header explicitly.

Dennis Hanuska November 24, 2015 18:19

It would be nice if it was the solution was that simple. Explicitly setting the character set in the header and even writing the received packet out to file made no difference. His problem appeared to be with the reception of the data. This problem is with the sending of the data. The reception works fine and when dumped out to a file correctly reflects the DN as it is stored in VTPP.

This does appear to be an encoding issue.

The RESTAPI does specify the UTF_8 character set in the headers.

Walter Goulet November 24, 2015 21:48

True, so I tackled the problem a different way and decided to try and the JSON payload before passing it to the Invoke-RestMethod call. By converting the JSON string from UTF8 to the Unicode encoding using the .NET System.Text.Encoding method, I was able to get the API to update the objects with names using extended ASCII characters. In my sample, the $app.DN is the DN of a basic application object with the long hyphen embedded in the name. If the sample works corrrectly, it will query the REST API to enumerate attributes of the application object. If Config/ReadAll succeeds (Result = 1), then it will use Config/WriteDN to set the 'Disabled' parameter to false.

Try this snippet and see if it works for you:

$uri = $VEDConnection.baseURI + '/Config/ReadAll?apikey=' + $VEDConnection.APIKey
$tmp = @{ObjectDN=$app.DN} | ConvertTo-Json
$json = [System.Text.Encoding]::Convert([System.Text.Encoding]::UTF8,[System.Text.Encoding]::Unicode,[System.Text.Encoding]::UTF8.GetBytes($tmp))
 
$results = Invoke-RestMethod -Uri $uri -Body $json -Method Post -ContentType 'application/json' -Verbose:$showRestMethodVerbose
if($results.Result -eq 1)
{
 $uri = $VEDConnection.baseURI + '/Config/WriteDN?apikey=' + $VEDConnection.APIKey
 $tmp = @{ObjectDN=$app.DN;AttributeName='Disabled';Values=@(0)} | ConvertTo-Json
 $json = [System.Text.Encoding]::Convert([System.Text.Encoding]::UTF8,[System.Text.Encoding]::Unicode,[System.Text.Encoding]::UTF8.GetBytes($tmp))
 $results = Invoke-RestMethod -Uri $uri -Body $json -Method Post -ContentType 'application/json' -Verbose:$showRestMethodVerbose
}

Dennis Hanuska November 25, 2015 15:15

Thanks, that solution worked in my environment as well.

Please sign in to leave a comment.