Overview

The Language Detection Anti-Spam filter is designed to block emails based on the detected language of the email body text.

In this article, you will learn how to determine why the Language Detection filter blocked or allowed a message as part of the troubleshooting process involving this Anti-Spam engine.

Introduction

The Language Detection filter determines the language of the email body text and is configurable to block certain languages. The module takes a portion of the email body message and analyzes it using the in-built language detection engine, currently able to identify over 130 different languages. The full list of supported languages is available in the Configuration UI under GFI MailEssentials > Anti-Spam > Anti-Spam Filters > Language Detection.

There will be scenarios where customers open support requests wanting to understand why the Language Detection engine blocked or allowed specific messages against their expectations. The next section outlines the troubleshooting process to determine the reason behind the actions taken by this filter.

Note: In order to correctly identify the language of the email body text, the first 5000 characters are sent to the language detection engine. The engine only processes email bodies with a minimum length of 50 characters.

Description

If you are questioning why an email was blocked or allowed by the Language Detection engine and would like more information, the best place to start the troubleshooting process is to examine the debug logs.

Follow the below procedure to find the log file and information regarding the email message under review, and thereafter use the examples provided to interpret and determine why the message was either blocked or allowed:

Find the Message-ID of the email in question by either obtaining it from the headers of the message itself or by looking for it in the MailEssentials Dashboard > Logs > Details tab. Refer to this linked article for more information on Reading Email Headers to extract the Message-ID.
Navigate to ..\GFI\MailEssentials\AntiSpam\DebugLogs and locate the log file for the Language Detection engine. The debug log filename is ase_language_detection.gfi_log.txt
- This debug log file for the module corresponds to GFI MailEssentials > Anti-Spam > Anti-Spam Filters > Language Detection on the configuration UI as well as LanguageDetection_Settings and LanguageDetection_Locales tables in the config.mdb database.
Open the debug log file in a text editor and search for the Message-ID obtained in step 1.
Refer to the log extracts below to determine the reasons behind the action taken by the module. Pay close attention to the lines in bold to understand what happened and why.

The debug log file will indicate whether the Language Detection engine is enabled and any actions taken by the filter while scanning emails. If the module is disabled the log file will show:

"ase_language_detection","======================================================================"
"ase_language_detection",">> CLanguageDetection::Init [Language Detection version: 21.6.11004.2]"
"ase_language_detection","Initializing db configuration..."
"ase_language_detection","Initializing registry configuration..."
"ase_language_detection","<< CLanguageDetection::Init()"
"ase_language_detection",">> CLanguageDetection::RefreshConfig()"
"ase_language_detection","Language Detection - Status: [Disabled]"
"ase_language_detection","Language Detection - Block Type: [Block the list below]"
"ase_language_detection","No locales are enabled."
"ase_language_detection","<< CLanguageDetection::RefreshConfig()"
"ase_language_detection","======================================================================"

Below log entries confirm that the Language Detection engine has successfully initialized as well as the currently configured language detection locales:

"ase_language_detection","======================================================================="
"ase_language_detection",">> CLanguageDetection::Init [Language Detection version: 21.6.11004.2]"
"ase_language_detection","Initializing db configuration..."
"ase_language_detection","Initializing registry configuration..."
"ase_language_detection","<< CLanguageDetection::Init()"
"ase_language_detection",">> CLanguageDetection::RefreshConfig()"
"ase_language_detection","Language Detection - Status: [Enabled]"
"ase_language_detection","Language Detection - Block Type: [Block the list below]"
"ase_language_detection","Adding locale [1089]"
"ase_language_detection","<< CLanguageDetection::RefreshConfig()"
"ase_language_detection",">> Init Message [<a94b225c8810cc8a7cf2db6341127c00@EC2AMAZ-TEDQDCP>]"
"ase_language_detection","Context Refreshed: No"
"ase_language_detection","Licensing check: Licensed"
"ase_language_detection","<< Init Message"
"ase_language_detection",">> Message Uninitialization"
"ase_language_detection","<< Message Uninitialization"
"ase_language_detection","----------------------------------------------------------------------"

The below debug log excerpt illustrates a successful language detection identified by the locale number 1033 (English US):

"ase_language_detection",">> CLanguageDetection::GetLocale"
"ase_language_detection","Locale [9]"
"ase_language_detection","SubLangID [0]"
"ase_language_detection","<< CLanguageDetection::GetLocale"
"ase_language_detection","Plain Text Locale [1033]"

Next, the provided locale number is searched in the config.mdb database, LanguageDetection_Locales table. A successful find looks as below:

"ase_language_detection",">> IsLocaleFound [1033]"
"ase_language_detection","<< IsLocaleFound [1033] - [YES]"
"ase_language_detection","Locale FOUND and Block Below List...returning true"
"ase_language_detection","Setting actions data ..."
"ase_language_detection","Spam detected, Stopping ASE Chain [2]..."
"ase_language_detection","Setting block report to: 'Blocked with language detection'"

The action taken is “Block below list”. The Language engine also provides "Block all except the list below" as an option if desired.

An exhaustive list of Windows Locale IDs (LCIDs) is available at this external link: Windows Locale Codes.

Back to top

Articles in this section

Determining why Language Detection Engine blocked or allowed a message

Overview

Introduction

Description

Related Articles