Read and search through a binary file

Robertico

Is it possible to read and search through a binary file.
I've a binary file an i'd like to search for certain hex values (markers).
After that i need to read eight bits just before the previous founded values.
(It's not a structured file)

Thanks in advanced,

Robertico

Steven Murawski

Robertico,

This might get you started -
http://www.microsoft.com/technet/scr.../payette2.mspx

It's an excerpt from Windows Powershell in Action and there is a
Get-HexDump function and some other info that could get you started.

Both Get-Content and Select-String can read from binary files.

Steven Murawski
Steven Murawski

Robertico wrote:
> Is it possible to read and search through a binary file.
> I've a binary file an i'd like to search for certain hex values (markers).
> After that i need to read eight bits just before the previous founded values.
> (It's not a structured file)
>
> Thanks in advanced,
>
> Robertico

Robertico

Steven,

So far i've this:

Get-Content -Encoding byte $path `
|%{ " " + ("{0:x}" -f $_).PadLeft(2,"0")

It generates a list with hex values. (i needed to convert from dec to hex)
How can i search for example: 131B1B087C156108AE151B
After that, i need the fileoffset of the result to read teh previous bits.

Robertico

"Steven Murawski" wrote:

> Robertico,
>
> This might get you started -
> http://www.microsoft.com/technet/scr.../payette2.mspx
>
> It's an excerpt from Windows Powershell in Action and there is a
> Get-HexDump function and some other info that could get you started.
>
> Both Get-Content and Select-String can read from binary files.
>
> Steven Murawski
> Steven Murawski
>
> Robertico wrote:
> > Is it possible to read and search through a binary file.
> > I've a binary file an i'd like to search for certain hex values (markers).
> > After that i need to read eight bits just before the previous founded values.
> > (It's not a structured file)
> >
> > Thanks in advanced,
> >
> > Robertico
>

Kiron

Try this:

# v1
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = [string]::join('', (gc $file -en byte | % {'{0:x2}' -f $_}))
[regex]::matches($bytes, $pattern) |
% {
$i = $_.index - $prevBytes * 2
[string]::join('', $bytes[$i..($i + $prevBytes * 2 - 1)])
}

# v2 CTP
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = (gc $file -en byte | % {'{0:x2}' -f $_}) -join ''
select-string $pattern -inp $bytes -all |
% {$_.matches |
% {
$i = $_.index - $prevBytes * 2
$bytes[$i..($i + $prevBytes * 2 - 1)] -join ''
}
}

--
Kiron

Kiron

If I misunderstood your question and you only need to check the previous byte's bits, try this:

# v1
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$bytes = [string]::join('', (gc $file -en byte | % {'{0:X2}' -f $_}))
[regex]::matches($bytes,$pattern) |
% {
$i = $_.index - 2
$byte = [string]::join('', $bytes[$i++..$i])
[convert]::toString(([int]"0x$byte"), 2)
}

# v2 CTP
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$bytes = (gc $file -en byte | % {'{0:X2}' -f $_}) -join ''
select-string $pattern -inp $bytes -all |
% {$_.matches |
% {
$i = $_.index - 2
$byte = $bytes[$i++..$i] -join ''
[convert]::toString(([int]"0x$byte"), 2)
}
}

--
Kiron

Robertico

v1) Is just what i needed.
Apologies for mistaken the use of bits / bytes.
I needed the previous bytes :-))

I discovered that [regex]::matches($bytes, $pattern) is case sensitive !

v2) give's an error on this line
$bytes = (gc $file -en byte | % {'{0:x2}' -f $_}) -join ''
Error: "You must provide a value expression on the right-hand of the
'-'-operator"

Robertico

"Kiron" <Kiron@HighPlainsDrifter.com> wrote in message
news:BE8CE4C6-11F8-45A4-A8B7-B1839382A1CD@microsoft.com...
Try this:

# v1
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = [string]::join('', (gc $file -en byte | % {'{0:x2}' -f $_}))
[regex]::matches($bytes, $pattern) |
% {
$i = $_.index - $prevBytes * 2
[string]::join('', $bytes[$i..($i + $prevBytes * 2 - 1)])
}

# v2 CTP
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = (gc $file -en byte | % {'{0:x2}' -f $_}) -join ''
select-string $pattern -inp $bytes -all |
% {$_.matches |
% {
$i = $_.index - $prevBytes * 2
$bytes[$i..($i + $prevBytes * 2 - 1)] -join ''
}
}

--
Kiron

Robertico

I needed to convert the returning values [string] to a little-endian value.

This doesn't work:
[bitconverter]::ToString([bitconverter]::GetBytes($out)) -replace '-',''
-or-
[bitconverter]::ToString([bitconverter]::GetBytes(0x$out)) -replace
'-',''

Appreciate some help.

Robertico


"Kiron" <Kiron@HighPlainsDrifter.com> wrote in message
news:BE8CE4C6-11F8-45A4-A8B7-B1839382A1CD@microsoft.com...
Try this:

# v1
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = [string]::join('', (gc $file -en byte | % {'{0:x2}' -f $_}))
[regex]::matches($bytes, $pattern) |
% {
$i = $_.index - $prevBytes * 2
[string]::join('', $bytes[$i..($i + $prevBytes * 2 - 1)])
}

# v2 CTP
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = (gc $file -en byte | % {'{0:x2}' -f $_}) -join ''
select-string $pattern -inp $bytes -all |
% {$_.matches |
% {
$i = $_.index - $prevBytes * 2
$bytes[$i..($i + $prevBytes * 2 - 1)] -join ''
}
}

--
Kiron

Kiron

Sorry Robertico, the hex formatting of the bytes was my fault but you can easily fix it by replacing the lowercase 'x' to an uppercase 'X', or by adding the 'IgnoreCase' option --or its numerical value 1-- to the [RegeEx]::Matches method, this is safer.
My system is LittleEndian, I get the same output after adding the extra code and removing the separating hyphens as without the extra code. Also the second code works on PowerShell version 2.0 CTP, if you're running version 1.0 it won't work, I added it just in case you had version 2.0 CTP.

Anyway try it and let us know if the output is what you're looking for.

# change te case of 'x' in this line:
$bytes = [string]::join('', (gc $file -en byte | % {'{0:x2}' -f $_}))
# to...
$bytes = [string]::join('', (gc $file -en byte | % {'{0:X2}' -f $_}))

# add the 'IgnoreCase' option in this line:
[regex]::matches($bytes, $pattern, 'ignoreCase') |
# or its numerical value...
[regex]::matches($bytes, $pattern, 1) |

-< Here are the fixed code for the two versions of PowerShell: >-
# v1
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = [string]::join('', (gc $file -en byte | % {'{0:X2}' -f $_}))
[regex]::matches($bytes, $pattern, 1) |
% {
$i = $_.index - $prevBytes * 2
[string]::join('', $bytes[$i..($i + $prevBytes * 2 - 1)]) |
% {
$hexBytes = $_
$byteArray = 0..($hexBytes.length - 1) | ? {!($_ -band 1)} |
% {
$i = $_
# [char][int]
"0x$($hexBytes.subString($i,2))"
}
[bitConverter]::toString($byteArray) -replace '-'
}
}

# v2 CTP
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = (gc $file -en byte | % {'{0:X2}' -f $_}) -join ''
select-string $pattern -inp $bytes -all |
% {
$_.matches |
% {
$i = $_.index - $prevBytes * 2
$bytes[$i..($i + $prevBytes * 2 - 1)] -join '' |
% {
$hexBytes = $_
$byteArray = 0..($hexBytes.length - 1) | ? {!($_ -band 1)} |
% {
$i = $_
[char][int]"0x$($hexBytes.subString($i,2))"
}
[bitConverter]::toString($byteArray) -replace '-'
}
}
}

--
Kiron

Robertico

Kiron,

Thanks for the explanation. I'am running version 1.0.
I didn't notice the meaning off v1 and v2. I thought that is was just an
other approach.

I need to convert the returning 'prevoius bytes' from "Big Endian" to
'Little Endian'' . The binary file is in "Big Endian".
The code " [bitConverter]::toString($byteArray) -replace '-' " indeed
doesn't convert to "Little Endian".
So i need some advice to fix this.

Robertico



"Kiron" <Kiron@HighPlainsDrifter.com> wrote in message
news:C356B032-5EF9-4337-B341-4BECD0AB2C21@microsoft.com...
Sorry Robertico, the hex formatting of the bytes was my fault but you can
easily fix it by replacing the lowercase 'x' to an uppercase 'X', or by
adding the 'IgnoreCase' option --or its numerical value 1-- to the
[RegeEx]::Matches method, this is safer.
My system is LittleEndian, I get the same output after adding the extra code
and removing the separating hyphens as without the extra code. Also the
second code works on PowerShell version 2.0 CTP, if you're running version
1.0 it won't work, I added it just in case you had version 2.0 CTP.

Anyway try it and let us know if the output is what you're looking for.

# change te case of 'x' in this line:
$bytes = [string]::join('', (gc $file -en byte | % {'{0:x2}' -f $_}))
# to...
$bytes = [string]::join('', (gc $file -en byte | % {'{0:X2}' -f $_}))

# add the 'IgnoreCase' option in this line:
[regex]::matches($bytes, $pattern, 'ignoreCase') |
# or its numerical value...
[regex]::matches($bytes, $pattern, 1) |

-< Here are the fixed code for the two versions of PowerShell: >-
# v1
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = [string]::join('', (gc $file -en byte | % {'{0:X2}' -f $_}))
[regex]::matches($bytes, $pattern, 1) |
% {
$i = $_.index - $prevBytes * 2
[string]::join('', $bytes[$i..($i + $prevBytes * 2 - 1)]) |
% {
$hexBytes = $_
$byteArray = 0..($hexBytes.length - 1) | ? {!($_ -band 1)} |
% {
$i = $_
# [char][int]
"0x$($hexBytes.subString($i,2))"
}
[bitConverter]::toString($byteArray) -replace '-'
}
}

# v2 CTP
$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = (gc $file -en byte | % {'{0:X2}' -f $_}) -join ''
select-string $pattern -inp $bytes -all |
% {
$_.matches |
% {
$i = $_.index - $prevBytes * 2
$bytes[$i..($i + $prevBytes * 2 - 1)] -join '' |
% {
$hexBytes = $_
$byteArray = 0..($hexBytes.length - 1) | ? {!($_ -band 1)} |
% {
$i = $_
[char][int]"0x$($hexBytes.subString($i,2))"
}
[bitConverter]::toString($byteArray) -replace '-'
}
}
}

--
Kiron

Kiron

Robertico,
I'm no expert on Endianness but I understand the difference is in the order of the bytes, so maybe by reversing the $byteArray you get what you want. If not, I hope someone with more knowledge on Endianness provides the correct or a better method.

Try this:

$file = <file's path>
$pattern = '131B1B087C156108AE151B'
$prevBytes = 8
$bytes = [string]::join('', (gc $file -en byte | % {'{0:X2}' -f $_}))
[regex]::matches($bytes, $pattern, 1) |
% {
$i = $_.index - $prevBytes * 2
[string]::join('', $bytes[$i..($i + $prevBytes * 2 - 1)]) |
% {
$hexBytes = $_
$byteArray = 0..($hexBytes.length - 1) | ? {!($_ -band 1)} |
% {
$i = $_
"0x$($hexBytes.subString($i,2))"
}
[array]::reverse($byteArray)
[bitConverter]::toString($byteArray) -replace '-'
}
}

--
Kiron