FSRM-Anti-ransomware: some remarks

[NotNull]

In this thread @SparkyZ asked:

I hope you'll take a look at my first Git Hub project and give me some feedback.

As I don't have a GitHub account and mentioned thread isn't the place to discuss that: especially for @SparkyZ a new thread ..


I did a quick-scan of half of your code (well documented, btw) and have some remarks. Not all may be accurate, but maybe it can help you anyway

Everything instance
You mentioned that it wasn't possible to "talk to" an Everything instance using the SDK.
I guess there has to be a way as ES.exe - command-line tool to query the Everything database - has an -instance option to , well, talk to an Everything instance.


Regex
You convert the filepatterns from the json file to regex queries.
That is not necessary. You can also use (example) wfn:"*.id-*.[admin@fentex.net].money"
(wfn: is the whole file name macro; see Search sytax help page)


Combining patterns
I couldn't find in the code what query you are "feeding" Everything, but you can combine multiple queries.
Using the wfn: synatx:
wfn:<"*.id-*.[admin@fentex.net].money"|"*.LOCKED_PAY">

I guess that will make querying faster


Order of execution
I would use this order to optimize waiting time:\
- Start Everything
- do your pattern chacks/manipulations
- Wait for/check if Everything available
- Start querying


Running Everything
Everything will try to re-use a running Everything.exe. Even when you spoecify the -admin option, it will re-use an Everything that is running under restricted user credentials ( IIRC for some tests I did a while ago).


False positives
The pattern list generates quite a few (80+) false positives on my system. Mostly Miscrosoft-signed files and a lot of *.info files.



I haven't actually tested your code, but wrot a quick-and-dirty script to demonstrate the stuff I mentioned earlier:
(that makes it probably more clear than my textual explanation)

FSRM.ps1

Code: Select all

#__________________________________________________________
#
#	SETTINGS
#__________________________________________________________
#

# How many patterns at once?
# Max length is (probably) 8191 so with 200: average pattern length 40 allowed
  $bulk = 200

# Where can the pattern list be downloaded
  $download = 'https://fsrm.experiant.ca/api/v1/combined'

  
#__________________________________________________________
#
#	Action!
#__________________________________________________________
#
  $jsonfile = (Invoke-WebRequest $download).Content

  $jsonobject = ConvertFrom-Json $jsonfile

  $patterns = $jsonobject.filters
  
# debug
  $patterns | out-file ".\patterns.txt"


  
 $start = 0 
 $end = $bulk -1

 While ($start -le $patterns.Count)
 {
	$query = ""

	# lazy mode: leave the lonely "|" at the beginning as wfn: will nullify that.

	$patterns[$start..$end] | % {$query = "$query|`"$_`""}
	$query = "wfn:<$query>"
	 
     # ES.exe is assumed to be in the %PATH%; otherwise: specify full path
	ES.exe $query
	
	$start = $start + $bulk
	$end = $end + $bulk
  }

[SparkyZ]

Thanks for the thoughts. I appreciate it very much. I have a lot to learn.

The actual query I'm feeding to Everything isn't regex. It's just dos-ish wildcards. The regex I mentioned is applied elsewhere. It's a mechanism to summarize and generalize all those specific ransomware filespecs. For example there's a billion (almost) xiaobao related filespecs. The AntiransomwareFiltersMerge.py application will find everything that matches "^\\*\\.[Xx]iao[Bb]a[0-9]*$" and summarize it "*.xiaoba*". The Everything search is essential to help people find false positives after applying those types of summarizations in the JSON file filter list.

My comment about the instance is too vague and I will rewrite it. I meant that using the "SDK" dll couldn't connect to a named instance of Everything. My goal would be to avoid talking to a currently running instance because its settings may not be general enough. My goal is to create a new instance that will see all the files. That's why I don't want it to read a configuration of any kind. That's also why I have that weird 10 second startup loop. In the future It will test to see if Everything is already running and then create a new instance. I probably need to switch to decaf and rethink that.

I didn't realize that I could combine query patterns. I'll poke that bear and see what happens. I will be impressed if I can dump all 3700 plus into one command. Currently I'm using 'utf8:case:file:noregex:ww:wfn:"'+fspec+'"' as the command and looping through each filespec one at a time.

A note about your ConvertFrom-Json: It totally made me crazy at first. Sometimes it would parse and other times blow up when I was using PowerShell 3 and 4 (no troubles with 5 and above though). I suggest using the "-Raw" flag so that the entire JSON loads into a single string rather than an array of strings. I have to temper that with the fact that this is my first PowerShell script over 5+ lines, so I may not know what I'm talking about at any point of this conversation. And I hate powershell at the moment.

I will tweak the order of operations. It will lead to a bit faster execution.

Thanks again for your thoughts.

[NotNull]

I have to add that my message was written in a race against the clock (not complete and even more typo's than normal) so sorry for misunderstandings.

One of the points I didn't express clearly enough was the part about instances. I actually *did* mean named instances.
ES.exe was used as an example as it can connect to a named instance. I don't know how, but if you download ES (from the downloads page), it comes with it's source code (I'm not a developer).

A note about your ConvertFrom-Json: It totally made me crazy at first. Sometimes it would parse and other times blow up when I was using PowerShell 3 and 4 (no troubles with 5 and above though). I suggest using the "-Raw" flag so that the entire JSON loads into a single string rather than an array of strings.

Thanks for that tip!

And I hate powershell at the moment.

Ha ha! I saw that in the code-comments too
The PowerShell script was just a demo of the "algorithm" I had in mind.
As English is not my native language, code can say it often much easier (less misunderstanding).

Good luck!

[SparkyZ]

No worries my friend. I'm sure we all had to race against the clock today and your English is just fine.

Thanks again for the encouraging words.