C++11 Signature Scanning

[atom0s]

One of the most well known and used methods of signature scanning is the FindPattern implementation from dom1n1k and Patrick of GameDeception. That method looks like this:

Code: [Select all] [Expand/Collapse]

1. bool Compare(const BYTE* pData, const BYTE* bMask, const char* szMask)
2. {
3.     for (; *szMask; ++szMask, ++pData, ++bMask)
4.         if (*szMask == 'x' && *pData != *bMask)   return 0;
5.     return (*szMask) == NULL;
6. }
7. DWORD Pattern(DWORD dwAddress, DWORD dwLen, BYTE *bMask, char * szMask)
8. {
9.     for (DWORD i = 0; i < dwLen; i++)
10.         if (Compare((BYTE*)(dwAddress + i), bMask, szMask))  return (DWORD)(dwAddress + i);
11.     return 0;
12. }

Here is a more modern method that I made and fine-tuned with a friend (devnull) and used in various projects of mine:

First, this is how I collect the data (externally) for usage with this method. We read the block of memory we want to scan then convert it into a vector.

Code: [Select all] [Expand/Collapse]

1. // Read the module into local memory..
2. auto data = new unsigned char[this->m_FFXiMainSize + 1];
3. ::ReadProcessMemory(handle, (LPVOID)this->m_FFXiMainBase, data, this->m_FFXiMainSize, NULL);
4. std::vector<BYTE> rawdata(data, data + this->m_FFXiMainSize);
5. delete[] data;

Note: This same method can be done internally without needing ReadProcessMemory! Simply just by setting the vector to use the base of FFXiMain.dll instead of the base of 'data'.

Now the implementation of our scanning method is as follows:

Code: [Select all] [Expand/Collapse]

1. /**
2.  * @brief Scans a given chunk of data for the given pattern and mask.
3.  *
4.  * @param data          The data to scan within for the given pattern.
5.  * @param baseAddress   The base address of where the scan data is from.
6.  * @param lpPattern     The pattern to scan for.
7.  * @param pszMask       The mask to compare against for wildcards.
8.  * @param offset        The offset to add to the pointer.
9.  * @param resultUsage   The result offset to use when locating signatures that match multiple functions.
10.  *
11.  * @return Pointer of the pattern found, 0 otherwise.
12.  */
13. static DWORD __stdcall FindPattern(std::vector<unsigned char> data, unsigned int baseAddress, const unsigned char* lpPattern, const char* pszMask, int offset, int resultUsage)
14. {
15.     // Build vectored pattern..
16.     std::vector<std::pair<unsigned char, bool>> pattern;
17.     for (size_t x = 0; x < strlen(pszMask); x++)
18.         pattern.push_back(std::make_pair(lpPattern[x], pszMask[x] == 'x'));
19.  
20.     // The result count for multiple results..
21.     auto resultCount = 0;
22.     auto scanStart = data.begin();
23.  
24.     while (true)
25.     {
26.         // Search for the pattern..
27.         auto ret = std::search(scanStart, data.end(), pattern.begin(), pattern.end(),
28.             [&](unsigned char curr, std::pair<unsigned char, bool> currPattern)
29.         {
30.             return (!currPattern.second) || curr == currPattern.first;
31.         });
32.  
33.         // Did we find a match..
34.         if (ret != data.end())
35.         {
36.             // If we hit the usage count, return the result..
37.             if (resultCount == resultUsage || resultUsage == 0)
38.                 return (std::distance(data.begin(), ret) + baseAddress) + offset;
39.  
40.             // Increment the found count and scan again..
41.             ++resultCount;
42.             scanStart = ++ret;
43.         }
44.         else
45.             break;
46.     }
47.  
48.     return 0;
49. }

The first part of our implementation is to build the pattern into a paired vector. This takes our pattern and mask and makes a vector of paired values holding the pattern to compare the data against. We do this because std::search allows us to pass an enumerable haystack and needle to look for. This makes things very easy to handle both the pattern byte and mask to compare against all in one go. The value pair is the byte of the pattern, then the boolean if the pattern byte is a wildcard or not based on the mask. (The mask implementation is based on the first example above, so we use 'x' as a required character, and anything else as a wildcard.)

The next step is our loop to call std::search. This is looped because we allow multiple matches to be found on the same signature. Sometimes, a pattern may match multiple functions and the first found match is not what is wanted. This is where the resultUsage parameter comes in. This allows us to define what match to return, if we want to use multiple matches.

std::search is setup to to take the following parameters:
- Iterator to determine the starting offset in the data to begin searching at. (Haystack start.)
- Iterator to define the end of the data we are scanning. (Haystack end.)
- Iterator to define the start of our pattern. (Needle start.)
- Iterator to define the end of our pattern. (Needle end.)
- Function to handle the comparison of the needle to the current haystack data.

Because we are making use of multiple results, scanStart is stored and used as the start of the haystack in case the user wants a different result other then the first. scanStart is set to the pattern+1 offset inside of the data each time it is found if the resultUsage is set.

Our comparison function is a simple lambda that checks our pattern object to determine if the current part of the pattern matches the current data from the haystack. Our first check determines if the pattern byte is a wildcard by seeing if its true (required). The second check is to ensure the byte of the pattern matches the current byte of the haystack.

Threaded Scanning With std::async!
The next step is to implement asynchronous scanning through C++11's async and shared_future features. You can read more into these here:
async: http://en.cppreference.com/w/cpp/thread/async
shared_future: http://en.cppreference.com/w/cpp/thread/shared_future

To start, we need to realize the difference between std::future and std::shared_future. We make use of std::shared_future because of a "limitation" with std::future. (And I say limitation in quotes because it is not really a limitation.) std::future is a single-shot async object. This means that once the method .get() is called once, the object is done. .get() cannot be called again on the object or an exception will be raised. Instead, we make use of std::shared_future because it keeps the result alive until the future is removed from scope. This means we can call .get() as many times as we want on the object and it will remain valid until the future is deleted / removed from scope.

We can store our pointers into a map that allows us to lookup the pointer later by name. This is in my opinion the easiest method of retrieving the pointers at a later time when needed. So our map would be like this:

Code: [Select all] [Expand/Collapse]

1. std::map<std::string, std::shared_future<unsigned long>> m_Signatures;

The next step is to load our signatures into a future within this map. We can do this like this:

Code: [Select all] [Expand/Collapse]

1. this->m_Signatures["sigName"] = std::async(std::launch::async, &FindPattern, std::ref(rawdata), sizeOfData, signature, mask, offset, resultUsage);

- std::ref(rawdata) is a reference to our vector of data dumped to scan within. (Haystack.)
- sizeOfData is the length of rawdata.
- signature is the signature to scan for.
- mask is the mask to compare against.
- offset is the offset to add after the pattern is found.
- resultUsage is the result to use if multiple results are known.

The last step is to validate the futures and ensure they completed before we attempt to use them. For this, we need to ensure the future's task is completed. For that we can do the following:

Code: [Select all] [Expand/Collapse]

1. // Ensure all futures are completed..
2. std::for_each(this->m_Signatures.begin(), this->m_Signatures.end(), [](std::pair<std::string, std::shared_future<unsigned long>> s)
3. {
4.     // Obtain the current future status..
5.     auto status = std::future_status::timeout;
6.     do
7.     {
8.         status = s.second.wait_for(std::chrono::milliseconds(5));
9.     } while (status != std::future_status::ready);
10.  
11.     // Obtain the status value..
12.     auto pointer = s.second.get();
13.    
14.     //
15.     // At this point you can check if pointer is valid and handle
16.     // any invalid pointers as needed. Perhaps you want the application
17.     // to fail to load if any pointers are invalid etc.
18.     //
19. });

And that's it! The last bit is just obtaining a signature that we want at another point in time. That can be done like this for example:

Code: [Select all] [Expand/Collapse]

1. /**
2.  * @brief Returns a pointers current value.
3.  *
4.  * @param name          The name of the pointer to obtain.
5.  *
6.  * @return The value of the pointer, 0 if not found.
7.  */
8. unsigned long Memory::GetPointer(const std::string& name) const
9. {
10.     auto pointer = this->m_Signatures.find(name);
11.     if (pointer == this->m_Signatures.end())
12.         return 0;
13.     return pointer->second.get();
14. }

[mr.exodia]

Nice stuff!

Personally I think the way of pattern matching you are using is very inconvenient, you have to keep track of different masks and pattern bytes.

Some time ago I wrote a similar solution (without async, but the type of pattern is pretty much the same):
https://bitbucket.org/mrexodia/patternfind

Here is both our efforts combined (C++11 removed and fully portable C++):
https://gist.github.com/mrexodia/f058868b81b2f1cb011a

Here is a copy of the new usage pattern:

Code: Select all

#include "FindPattern.h"
#include <iostream>
 
using namespace std;
 
int main()
{
    unsigned char test[16] = { 0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF };
    vector<unsigned char> data(test, test + sizeof(test));
    size_t result = FindPattern(data, "?7 8? ?? CD");
    cout << "result: 0x" << hex << result << endl;
    return 0;
}

[atom0s]

Personally I think the way of pattern matching you are using is very inconvenient, you have to keep track of different masks and pattern bytes.

I'm not quite sure what you mean by this. Looking at your method you are doing the same thing by needing a signature and a mask, so there is no real difference there. The point of my g_Signatures global array thing is how it is used in a personal project of mine. Since the library is released compiled, I keep the signatures stored in a global array for use whenever the library is loaded. It can be used inline as well if I wanted to do it that way, but it was just easier to store them all in a single easy to edit header file instead.

It is not needed for this code to work though, it was mearly how I use it in one of my projects.

Some time ago I wrote a similar solution (without async, but the type of pattern is pretty much the same):
https://bitbucket.org/mrexodia/patternfind

Here is both our efforts combined (C++11 removed and fully portable C++):
https://gist.github.com/mrexodia/f058868b81b2f1cb011a

Here is a copy of the new usage pattern:

Code: Select all

#include "FindPattern.h"
#include <iostream>
 
using namespace std;
 
int main()
{
    unsigned char test[16] = { 0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF };
    vector<unsigned char> data(test, test + sizeof(test));
    size_t result = FindPattern(data, "?7 8? ?? CD");
    cout << "result: 0x" << hex << result << endl;
    return 0;
}

Thanks for your example too. I never personally needed bit specific scanning so it was never something I bothered to implement.

As for being portable, mine should be. It is using standard STL stuff which is part of C++. GCC for Linux has the best C++11 support than any compiler for the language as well, so it should all compile file on Linux too.

[mr.exodia]

I'm not quite sure what you mean by this. Looking at your method you are doing the same thing by needing a signature and a mask, so there is no real difference there. The point of my g_Signatures global array thing is how it is used in a personal project of mine. Since the library is released compiled, I keep the signatures stored in a global array for use whenever the library is loaded. It can be used inline as well if I wanted to do it that way, but it was just easier to store them all in a single easy to edit header file instead.

You have to keep track of the mask outside of FindPattern, separately from the data. This is what the example I implemented would look like with your code:

Code: Select all

int main()
{   
   //67 89 ?? CD (requires two variables)
   unsigned char pattern1data[] = { 0x67, 0x89, 0, 0xCD };
   char pattern1mask[] = "00x0";

   unsigned char test[16] = { 0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF };
   vector<unsigned char> data(test, test + sizeof(test));
   
   long result = FindPattern(data, 0, pattern1data, pattern1mask, 0, 0); //FindPattern is the original implementation
   cout<<hex<<result<<endl;
   return 0;
}

With mask and data integrated ('my' code):

Code: Select all

int main()
{
    char pattern1[] = "67 89 ?? CD"; //only one string needed

    unsigned char test[16] = { 0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF };
    vector<unsigned char> data(test, test + sizeof(test));
    size_t result = FindPattern(data, pattern1);
    cout << "result: 0x" << hex << result << endl;
    return 0;
}

In this way you only keep track of one string as signature, instead of one array of data and one string as mask, that's what I meant. In addition you have to be 100% certain your mask is not bigger than the pattern data because it will cause undefined behavior (the number of bytes you read from data is derived from the mask), with my solution you cannot go wrong there either

As for being portable, mine should be. It is using standard STL stuff which is part of C++. GCC for Linux has the best C++11 support than any compiler for the language as well, so it should all compile file on Linux too.

Your code uses only portable C++ functions, but DWORD is not a type defined in C++ (it's a winapi typedef) and your code would not work on most x64 executables (since most of them have a base > 0xFFFFFFFF meaning it would overflow the addition).

Greetings

[atom0s]

I'm not quite sure what you mean by this. Looking at your method you are doing the same thing by needing a signature and a mask, so there is no real difference there. The point of my g_Signatures global array thing is how it is used in a personal project of mine. Since the library is released compiled, I keep the signatures stored in a global array for use whenever the library is loaded. It can be used inline as well if I wanted to do it that way, but it was just easier to store them all in a single easy to edit header file instead.

You have to keep track of the mask outside of FindPattern, separately from the data. This is what the example I implemented would look like with your code:

Code: Select all

int main()
{   
   //67 89 ?? CD (requires two variables)
   unsigned char pattern1data[] = { 0x67, 0x89, 0, 0xCD };
   char pattern1mask[] = "00x0";

   unsigned char test[16] = { 0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF };
   vector<unsigned char> data(test, test + sizeof(test));
   
   long result = FindPattern(data, 0, pattern1data, pattern1mask, 0, 0); //FindPattern is the original implementation
   cout<<hex<<result<<endl;
   return 0;
}

With mask and data integrated ('my' code):

Code: Select all

int main()
{
    char pattern1[] = "67 89 ?? CD"; //only one string needed

    unsigned char test[16] = { 0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF };
    vector<unsigned char> data(test, test + sizeof(test));
    size_t result = FindPattern(data, pattern1);
    cout << "result: 0x" << hex << result << endl;
    return 0;
}

In this way you only keep track of one string as signature, instead of one array of data and one string as mask, that's what I meant. In addition you have to be 100% certain your mask is not bigger than the pattern data because it will cause undefined behavior (the number of bytes you read from data is derived from the mask), with my solution you cannot go wrong there either

As for being portable, mine should be. It is using standard STL stuff which is part of C++. GCC for Linux has the best C++11 support than any compiler for the language as well, so it should all compile file on Linux too.

Your code uses only portable C++ functions, but DWORD is not a type defined in C++ (it's a winapi typedef) and your code would not work on most x64 executables (since most of them have a base > 0xFFFFFFFF meaning it would overflow the addition).

Greetings

I totally miss-read your code and thought the test data was your signature for some reason lol. So ignore that part.

As for portability, yeah, some of the types like DWORD wont port but the STL code should be fine.

[atom0s]

Here is a version that should be both portable and 64bit compatible:

Code: [Select all] [Expand/Collapse]

1. /**
2.  * @brief Scans a given chunk of data for the given pattern and mask.
3.  *
4.  * @param data          The data to scan within for the given pattern.
5.  * @param baseAddress   The base address of where the scan data is from.
6.  * @param lpPattern     The pattern to scan for.
7.  * @param pszMask       The mask to compare against for wildcards.
8.  * @param offset        The offset to add to the pointer.
9.  * @param resultUsage   The result offset to use when locating signatures that match multiple functions.
10.  *
11.  * @return Pointer of the pattern found, 0 otherwise.
12.  */
13. static intptr_t FindPattern(std::vector<unsigned char> data, intptr_t baseAddress, const unsigned char* lpPattern, const char* pszMask, intptr_t offset, intptr_t resultUsage)
14. {
15.     // Build vectored pattern..
16.     std::vector<std::pair<unsigned char, bool>> pattern;
17.     for (size_t x = 0, y = strlen(pszMask); x < y; x++)
18.         pattern.push_back(std::make_pair(lpPattern[x], pszMask[x] == 'x'));
19.  
20.     auto scanStart = data.begin();
21.     auto resultCnt = 0;
22.  
23.     while (true)
24.     {
25.         // Search for the pattern..
26.         auto ret = std::search(scanStart, data.end(), pattern.begin(), pattern.end(),
27.             [&](unsigned char curr, std::pair<unsigned char, bool> currPattern)
28.         {
29.             return (!currPattern.second) || curr == currPattern.first;
30.         });
31.  
32.         // Did we find a match..
33.         if (ret != data.end())
34.         {
35.             // If we hit the usage count, return the result..
36.             if (resultCnt == resultUsage || resultUsage == 0)
37.                 return (std::distance(data.begin(), ret) + baseAddress) + offset;
38.  
39.             // Increment the found count and scan again..
40.             ++resultCnt;
41.             scanStart = ++ret;
42.         }
43.         else
44.             break;
45.     }
46.  
47.     return 0;
48. }

[poKebol]

Ty for your attention atom0s.

What if i don't have a mask? like Cheat engine, i don't need a mask to search.

[atom0s]

The mask is just a byte per byte basis. You apply it to the bytes you have in your pattern to search for.

For example, in Cheat Engine if you have search like:
89 BF 90 90 90 FF 90 ?? ?? ?? ?? 12 34 56

The mask for this would be:
xxxxxxx????xxx

Each x or ? is 1 byte of the pattern. So:
89 = x
BF = x
90 = x
90 = x
90 = x
FF = x
90 = x
?? = ?
?? = ?
?? = ?
?? = ?
12 = x
34 = x
56 = x

[poKebol]

Hi,

I'm trying to find a array of bytes using your scan code:

Code: [Select all] [Expand/Collapse]

1. DWORD WINAPI Scan(LPVOID lpParam)
2. {
3.     if (!m->setHandle())           
4.         return 0;
5.  
6.     Search* args = (Search*)lpParam;
7.  
8.     SYSTEM_INFO sysInfo = { 0 };
9.     GetSystemInfo(&sysInfo);
10.  
11.     auto aStart = (long)sysInfo.lpMinimumApplicationAddress;
12.     auto aEnd = (long)sysInfo.lpMaximumApplicationAddress;
13.  
14.     int found = 0; 
15.  
16.         while (aStart < aEnd)
17.         {
18.             MEMORY_BASIC_INFORMATION mbi = { 0 };
19.             if (!VirtualQueryEx(m->getHandle(), (LPCVOID)aStart, &mbi, sizeof(mbi)))
20.             {
21.  
22.                 CloseHandle(m->getHandle());
23.                 return 0;
24.             }
25.  
26.             if (mbi.State == MEM_COMMIT && ((mbi.Protect & PAGE_GUARD) == 0) && ((mbi.Protect == PAGE_NOACCESS) == 0))
27.             {
28.  
29.                 auto isWritable = ((mbi.Protect & PAGE_READWRITE) != 0 || (mbi.Protect & PAGE_WRITECOPY) != 0 || (mbi.Protect & PAGE_EXECUTE_READWRITE) != 0 || (mbi.Protect & PAGE_EXECUTE_WRITECOPY) != 0);
30.                 if (isWritable)
31.                 {
32.                     auto dump = new unsigned char[mbi.RegionSize + 1];
33.                     memset(dump, 0x00, mbi.RegionSize + 1);
34.  
35.                     ReadProcessMemory(m->getHandle(), mbi.BaseAddress, dump, mbi.RegionSize, NULL);                
36.  
37.                     BYTE data1[] = "\0x8D\0x01\0x00\0x00\0x00\0x00\0x00\0x00\0x01\0x00";
38.                     char mask[] = "xxxxxxxxxx";
39.                    
40.                     for (auto x = 0; x < mbi.RegionSize; x++)
41.                     {
42.                        
43.                         if (MaskCheck((BYTE*)(dump + x), data1, mask))
44.                         {
45.                             found++;
46.                             //BYTE data[] = { 0xF0, 0x49, 0x02, 0x00 , 0x00, 0x00, 0x00, 0x00, 0x01, 0x00 };
47.                             //DWORD wAddr = (DWORD)mbi.BaseAddress + x;
48.                             //WriteProcessMemory(m->getHandle(), (LPVOID)wAddr, &data, sizeof(data), NULL);
49.                         }
50.                     }
51.  
52.  
53.  
54.                     for (auto x = 0; x < mbi.RegionSize - 4; x += 4)
55.                     {
56.                         int price = (int)*(DWORD*)(dump + x);
57.                         if (price == 45000)
58.                         {
59.                             int newPrice = 0;
60.                             DWORD wAddr = (DWORD)mbi.BaseAddress + x;
61.                             WriteProcessMemory(m->getHandle(), (LPVOID)wAddr, &newPrice, sizeof(newPrice), NULL);
62.                         }
63.  
64.                         double sell = (double)*(DWORD*)(dump + x);
65.                         if (sell == 2250)
66.                         {
67.                             int newSell = 250000;
68.                             DWORD wAddr = (DWORD)mbi.BaseAddress + x;
69.                             WriteProcessMemory(m->getHandle(), (LPVOID)wAddr, &newSell, sizeof(newSell), NULL);
70.                         }                      
71.  
72.                     }
73.  
74.                     delete[] dump;
75.                 }
76.  
77.             }
78.             aStart += mbi.RegionSize;
79.  
80.             Sleep(1);
81.         }  
82.  
83.     CloseHandle(m->getHandle());
84.  
85.     std::string f = std::to_string(found);
86.     dw->SetStatusText(f.c_str());
87.     return 0;
88. }

Code: [Select all] [Expand/Collapse]

1. BOOL MaskCheck(const BYTE* lpData, const BYTE* lpPattern, const char* szMask)
2. {
3.     for (; *szMask; ++szMask, ++lpData, ++lpPattern)
4.         if (*szMask == 'x' && *lpData != *lpPattern)
5.             return FALSE;
6.     return (*szMask) == NULL;
7. }

What i'm trying to do is this:

type Array of Bytes, and search for: 8D 01 00 00 00 00 00 00 01 00
Select all and change to: F0 49 02 00 00 00 00 00 01 00

I don't have a mask, i set the mask to xxxxxxxx.
It never found. =(

[atom0s]

Your mask needs to be the same length as your array of bytes. In your case you have:
8D 01 00 00 00 00 00 00 01 00

This means your mask would be 10 bytes long:
xxxxxxxxxx