Tuesday, April 28, 2015

Tangential to the dominant paradigm - Introducing CyberMonkey

It's not a post on forensics (your first and only warning), but it is geekery ... with a slight banana flavor.

I just finished a small side project that has been growing in the background for a while. It's a mish mash of various electronic, hand made,and mechanical parts that I've accumulated overtime.

It has been evolving , and a couple of days ago I added in Bluetooth LE support ... and as you know everything is better with Bluetooth.

Recipe for Cyber Flying Monkey
- One wooden money with vest and fez
- 2 x EL Wire (different colors)
- 1 x Arduino (choose your flavor)
- 1 x Neopixel
- Wings
- Bluetooth LE module
- idle hands
- and mix well with random contiguous chunks of time.
- (some parts missing from recipe to protect the world from a legion of self aware evolving cyber monkeys.)

Most likely, cyber monkey, will evolve into some sort of Internet aware notification system. With his heart changing color to indicate different status updates. He may raise his wings, and flash his gun hand and fez to highlight important messages. I'm looking at pairing him with a Raspberry PI or BeagleBone over Bluetooth LE.

Monday, March 30, 2015

Registry Reporter - Beta Release

Almost an hour and a half after the official end of the day, but the Beta 0.1 of Registry Reporter has been officially released for testing. Please remember this is a beta, and therefore should not be used for production level work.

The beta release has been posted to the following URL:

If you find issues please send me an email to: erik@erikmiyake.us

  1. Describe the steps you took to cause the error.
  2. What hive and what key you were reading from.

In addition to errata, I would appreciate feedback on features and design.


Wednesday, March 25, 2015

Registry Reporter - Last post before beta release

If you did not see my prior post, I've renamed what I was calling "Registry Explorer" to be "Registry Reporter".  It's been a long road to this point. Many distractions slowed down the beta release timeline, including the addition of new features and behaviors into the "Registry Reporter" application.

I've been suffering from the Engineer's Dilemma: of when to release, when is it ready? To avoid further slippage on the beta release, I've drawn a line in the sand for myself of the end of this week.

So expect a post with download link by end of day .. March 29th, 2015 PST. There it has been said, and soon as this post is published it must be a reality.

Clarification: As I was writing, I realized that I was switching between being lazy and using abbreviations and spelling out the whole terms. Since the terms also appear in the application itself, I thought I would explain:

  • NK - Named Registry Key (visually represented as a folder icon in the application)
  • VK - Value Registry Key (visually represented as a file document icon in the application)

Here the are new features:

Main Interface
The main interface layout has remained basically the same, with the only alteration being the addition of the "Bookmarks" (upper left) and "VK Data Hex" (lower left) tabs. Most  of the additional features are available through drop down menus.

Table displays VK and NK.
The data table (picture aboce) which displays the registry keys, for the Named Key selected in the tree on the left hand side, now displays both the Value Key and Named Key. Previous it only displayed the Value Keys for the selected Name Key. This table allows you to drill down in the registry structure by double clicking on a Named Key in the table.

You can now bookmark the selected key in the data table. This allows you to identify keys of interest for reporting purposes and bookmarking them for later reference.

 The bookmarked registry key can be found under the "Bookmarks" tab. Selecting the bookmark will cause the data table to display key or keys children.

Modified Within
You can now select a key of interest (NK or VK) and populate the Timestamp tab with entries within 1Hr or 24Hrs of the selected NK key or selected VK key's parent.

Hex of Value
You can now view and interact with the hex data for the Value Keys raw data through a hex control (pictured above) under the "VK Data Hex" tab. This allows you to copy values, hash data selections, and much more. This hexcontrol only displays the data for the selected Value key.

Raw Hive - View
You can also view the selected key in situ within the hive itself (pictured above). This is launched through a drop down menu from the data table and the hexcontrol is displayed in a dialog window. The selected key is broken out into it's separate parts so you can see the internal structure. Placing the mouse cursor over a colored break out region causes a tool tip to display the selection name i.e. Slack (in the picture above).

Raw Hive - View
A limited data probe feature has been implemented. Manually selecting bytes will cause the data to be interpreted if the selection size matches the type i.e. a 2 byte selection populates the rows: Char. UInt16, and Int16. Clicking on a break out region causes that region to automatically be selected and the value displayed in the probe table. In the picture above, the cell size was clicked on and the value of the cellsize (-88) can be see in the Int32 row.

The following features are in the pipeline to be completed:

  1. Indicator of Compromise (IOC) 
    • Registry IOCs based on Mandiant OpenIOC format.
    • Online support for sharing registry IOC definitions.
    • Creation of IOCs based on the select key, or selected data.
    • IOC report generation.
    • Note: if anyone has examples of  RegistryIOCs that they currently use or have created. I would appreciate seeing them. 
  2. Timestamp Helper
    • Adding value keys with timestamps within the keys data to the timestamp filtering.
    • This feature allows items like UserAssist entries which can have an embedded timestamp to be used for timeline based activity tracking.
    • Users can define a custom helper for keys which have a fixed offset for the timestamp and where the key can be identified through a direct full path or regex full path match.
    • Registry value keys which have a variable offset will be support and developed as time allows and user interest warrants. 
  3. Saving Bookmarks  
    • saving bookmarks.
  4. Recovery Of Keys 
    • Key recovery has been written, but I've not taken the time to complete my due dilligence, to my satisfaction, so it will not be part of the initial beta release.

Wednesday, March 11, 2015

Forensic Framework - Teaser 3

Just finished posting the video for the third teaser. The feature updates below had eaten up time that was supposed to be allocated to the Registry Reporter (new name for what I was calling Registry Explorer). I'm getting back to the Registry Reporter this week, so expect a beta soon.

Below is the teaser 3:

Feature Updates:

Overlay Toolbox
- Treble Layout
- Editable Favorites
- Drag and Drop to the Hex Control
- Selectable palette
Hex Tools
- Dropper Select
- Paint Brush Select
- XRAY vision
-Selected color auto cycling
Data Probes
- GUI Rearrangement
Data Visualization
- Control overhaul
- Scrolling and paging updates.
- Zoom level buttons
- Display of selections in hex control
Hex Control
- added SeekUntil overlay feature
- modified to support tool behavior changes

Image: XRAY feature

User can enable the XRAY tool and select an overlay type to have the hex control highlight items which match the overlay type. Some overlay types are more discriminating than others, and are therefore more effective to identify data of interest i.e. "Windows 64 LE Timestamp" vs UInt64.

Image: Favorites toolbox editor

A user can customize a favorites toolbox with only the overlay types that they wish to work with at the time. These custom toolboxes can be saved and loaded later.

Image: Data probes

Data probes allow a user to selected an offset within the hex control and have it evaluated against different data types. The data probes are broken into three categories: Binary(Numeric), Timestamp, and Text Encoding.

Image: Binary probe

Image: Timestamp probe

The display formatting, timezone, and valid date ranges can be modified for the timestamp probe. The data range allows the timestamp probe control to only display decoded data time values which fall within the range of interest. This helps eliminate dates such as  3 AD, January 1rst  ...( I think that is a Monday)...  which probably doesn't have any relevance to your data.   
Image: Encoding probe

The visualization control has been overhauled to provide more intuitive controls, magnification, and paging/scrolling. 

Image: Visualization - gradient

Image: Visualization - High low

Image: Visualization - Unicode - English

Monday, January 26, 2015

Forensic Framework - Teaser 2

That first one, really wasn't much of teaser. Barely enough to get you interested. I'm hoping that this video helps to stir the curiosity a little more.

In the teaser above we are looking at an mp3 file. This teaser lightly highlights the visualization, hex display searching, text selection, and strings features.

Here are a couple of screen captures from the video:

Friday, January 9, 2015

Forensic Framework - Teaser 1

I've been working a forensic application for quite a time now. More time than I would like to admit; as work called me away, life interrupted, or my interest waned for a period of time. It has mainly been a testing ground for my ideas, and frustrations .. when whatever main stream tool I was using failed or crashed.

I'm publicly referring to it as the generic name "Forensic Framework" as I decide whether I'm going to commercialize it or not. Here is the crash list of features. I'm sure I've overlooked a couple. I have made the attempt to line it up under existing, partial/in-progress, or planned. Not a finalized list; it may undergo some shifting later as I shake out the broken bits, or as features get pushed down in favor of others.

I’ll have a follow up post with pictures later. I’m working on a clean up of the Registry Explore code this weekend, so may not happen until next week.

It supports the following features:
  • Supported Media
    • Reading EWF Images (multi-segment, and verification)
    • Reading DD Images (monolithic, and multi-segment)
    • Reading Physical Media
    • Reading local files and folders.
  • FileSystem
    • FAT
      • FAT12
      • FAT16
      • FAT32
    • Deleted Files
    • Volume Recovery
      • Scans for NTFS and FAT
      • Recovery of FAT
  • Hex Display
    • Changes to encodings for display.
    • Searching content
    • Seek until
      • Run end
      • Zero
      • Printable ASCII
      • Not printable ASCII
  • Hashing
    • MD5
    • SHA1
    • SHA256
    • SHA384
    • SHA512
  • Entropy Calculation
  • Intelligent selection of data
    • JSON Objects
    • ASCII Printable
    • Base64
    • MFTRecord
    • PST Compressible Text
    • Unicode BE Arabic
    • Unicode BE
    • Unicode BE English
    • Unicode LE English
    • Email Address
    • .. and many more around the corner
  • Strings
    • ASCII
    • GSM03.38
    • KOI8 R Cyrillic
    • UTF-8
    • UTF-16
      • Arabic
      • Armenian
      • Cyrillic
      • + 27 other character ranges.
    • PST Strings (Outlook)
      • UTF-16 (see UTF-16 above)
      • ASCII
    • ISO 
      • Thai, Portugese. Romanian, Czech, Polish, Serbian, Slovene, Turkish, Cyrillic
    • CP 866 - Cyrillic
    • Compressed Unicode
      • Arabic 
      • Cyrillic
      • +10 other character ranges
  • Data Visualization
    • Drive Map (sectors)
    • Binary data with color value per byte
  • Bookmarking
    • Images (PNG,TIF,BMP,JPG, GIF, EMF, WMF, XBM, Base64Encoded)
    • File/Files 
    • Overlays
      • Includes text encodings
  • Mountable Files
    • Nokia Backup.arc
  • Overlays
    • Use_Encoding,
    • Byte,Int 16Bit ,Int 32Bit , Int 64Bit,UInt 16Bit, UInt 32Bit, UInt 64Bit,
    • Double 64Bit , Char UTF16, Single Float, 
    • Hex, Binary
    • ROT5, ROT13, ROT18, ROT47,
    • ASCII_7Bit, Base64 , QuotePrintable, ASCIIPrintable
    • Raw_Binary
    • YahooBase64, YahooUsernameEncoding  
    • URL, URLEncoding, StripHTMLTags,  HTMLEntities,
    • CompressedUnicode, MP3Frame
    • PSTCompressibleASCII, PSTCompressibleUnicode,
    • GUID,
    • MFTEntry, FATVBR, NTFSVBR, DOSPartitionEntry, DOSPartitionTable, FATDirectoryEntry
    • BPList Object,
    • MAC Absolute Time, Windows64bit  date time, FileTime, MS-DOS date time, Unix time, Symbian time
    • NokiaNibbleCounter, SwapByteEncodedPhoneNumber
  • Phone
    • OBEX support via BT (code may be removed)
    • PM Flash Files 
    • Blackberry IPD Records
  • File Carving
    • User defined file signature
  • Intelligent Carving/Recovery
    • AMR files
    • Flat Decoded Streams
    • Deflate Streams
  • Signature Analysis
  • HTML Reporting
  • Binary Probe
    • LE and BE versions of standard variables (Int, Double, etc…)
  • Timestamp Probe
    • HTMLFileTime
    • MacAbsolute
    • MSDOS32Bit
    • PRTime
    • A lot more planned but not implemented/tested
  • Encoding Probe
    • Base64
    • ROT13
    • ROT47
    • Outlook ASCII
    • Outlook Unicode
    • UTF-8
    • UTF-16
  • MetaData
    • EXIF
      • EXIF GPS Extraction to KML
    • EXE
    • BMP
    • DOC
    • FLV
    • GZ
    • IPD Record
    • SQLite
    • LNK
  • OS Artifacts
    • Windows
      • Registry Files
    • EVT Parsing/Carving
    • Macintosh
      • BPLIST file
      • COOK file

It has the following features in progress or partially supported (some may not make the final program):
  • FileSystem
    • NTFS
  • Mountable Files
    • Compound Structure File Format (doc, msi, thumbs.db,)
    • PDF
    • Zip Compressed
  • Data Visualization
    • TreeMaps
      • Data size/Type
      • Date Created/Accessed/Modified
  • MetaData
    • MOV 
  • Scripting  (most likely will be dropped).
    • Iron Python
    • C#/.NET
  • Carving
    • PST Content Carving
  • Batch Searching
Planned Features 
  • FileSystem
    • EXFAT
  • Advanced Data Recognizers (analyze content i.e cluster, segment, to determine data types)
    • Unicode by Language
    • JPEG

 If you made it this far, here you go .. one picture. The only picture in the wild.

Saturday, January 3, 2015

EDC - Write blocker Verification

Within forensics protecting your original media is of the highest importance. Being able to verify that your write blocker is functioning is a requirement for all laboratories, office practitioners, and wandering semi-nomadic analysts.

  • How do you know your write blocker is functioning? 
  • When was the last time it was verified as functional? 

Downside of all this is it can be fairly time consuming to perform a verification on your write blocker. For one write blocker you end up performing a minimal of two hashings of the media for the simplest verification process.  A single pass hash of a modern hard drive can take hours, multiply that by two and that machine is down for the day. If your agency or organization has a more rigorous process, then it can be a much larger time sink.

Additional down sides:

  • Hard drives are heavy.
  • Hard drives are fragile. 

What are the solutions to making this process quicker.

  • Use a faster hard drive.
  • Use a smaller hard drive. 

Solution: Don't use a hard drive. Use flash media.

  • Flash media is fast. 
  • Flash media is small and portable.
  • Flash media is much more sturdy than a physical platter hard drive.

I have as part of my portable kit the following solution:

  1. Compact flash to SATA adapter.
  2. Compact flash to IDE adapter.
  3. Several small CF Cards 1GB

Transcend 1GB 133x compact flash card

CF card with SATA adapter attached to write blocker.

CF card with IDE adapter attached to write blocker.

Pictured above is the elapsed time from hashing the media captured just before hitting 100%.  A 1GB card took approximately one minute to hash. If you are hashing twice, then two minutes plus some change for your attempts to change the media. If you have to verify the 10 built-in write blockers in the machines in your lab, plus the four portable write blocker kits, what was once a grueling slog through the verification process has become a simple task. 

When I started to use this solution for testing I was using both compact flash media and SD card media. It worked fine at the time. Eventually, I rebuilt my kit and purchase new adapters. I started to have problems with the length of time it as taking to hash. Inexplicably this quick short procedure suddenly be came a long painful process, sometimes stalling out. So I stopped using the kit and switched back to the platter based hard drives for a while. A couple of write blocker firmware upgrades later, and a retooling of the kit, and I'm back to using it.

With the latest iteration of my kit I've gone to only using CF flash cards.  A compact flash card  is an ATA device with an ATA controller and ATA compatible electrical interface.  Commands should be simply passed through, with the adapter mainly being passive. An SD card SATA or IDE adapter has to translated the instructions from the ATA protocol/interface to the SD card protocol/interface. 

Simply put, a CF card speaks native hard drive and the SD card requires a translator.

As always, verify and validate your tools. There are many different adapter vendors out there, some have worked for me others have tried my patience until the product was stress tested in an aggressive manner.