[FortiGate] Understand the basic settings of the Web filter function

Firewall (UTM)
2021.10.30

Work environment

  • FortiGate 60E
    • version 7.0.1

Web filter function overview

The Web Filter feature allows you to apply a Web Filter profile to a (firewall) policy to limit or control user access to Web resources for traffic that matches that policy according to the contents of the applied profile.

  • Security profile setting field on the policy setting screen

Three major components

  1. Web content filter
    • Block web pages that contain the specified word or pattern
  2. URL filter
    • Use URLs and URL patterns to block or allow web access to matching URLs, or block malicious URLs detected by FortiSandbox.
  3. FortiGuard Web Filtering Service
    • Provides many additional categories that can be used to filter web traffic, enabling categorical access control

Default web filter profile

By default, there are three web filter profiles:

  • default
  • monitor-all
  • wifi-default

You can customize and use these profiles, or you can create and use new original profiles.

Web filter profile setting screen

If you set it from the GUI, you can do it on the [Security Profiles > Web Filter] screen.

To create a new profile, click the [Create New] button.

Flow-based and Proxy-based

You can select flow-based or proxy-based as the value of [Feature set], which is a setting item of the profile.

There are some features that can only be used with proxy-based, but proxy-based is said to have lower throughput than flow-based.

◆Setting items for flow-based

◆Setting items for proxy-based

The setting items marked with (P) in the image above are functions that can be used only when proxy-based.

Config in CLI

The config of the web filter profile in the CLI is config webfilter profile.

config webfilter profile
    edit "default"
        set comment "Default web filtering."
        set feature-set proxy
        config ftgd-wf
            unset options
            config filters
                edit 1
                    set category 2
                    set action block
                next
                edit 2
                    set category 7
                    set action block
                next
                edit 3
                    set category 8
                    set action block
                next
                ...
                edit 23
                    set category 91
                    set action block
                next
            end
        end
    next
end

Web filter function application order

The order in which it is applied for multiple web filter functions is as follows.

  1. URL filter
  2. FortiGuard Web Filter
  3. Web content filter
  4. Web script filter
  5. Antivirus scan

By using this application order, you can design the URL filter to exclude only the specified URL from the subsequent filtering.

URL filter settings

URL filters allow you to block, allow, monitor, and so on specific URLs that have a pattern that contains the specified text or regular expression.

URL filters are set in the [Static URL Filter > URL Filter] field on the Web filter profile setting screen.

URL filter type

  • Simple:
    • Judge as a hit if it exactly matches the specified URL
  • Regular Expression/Wildcard:
    • Determine based on regular expression or wildcard rules

URL filter action

  • Exempt:
    • Exempted from evaluation by subsequent Web filter functions (Web content filter, etc.) (permitted)
    • Used with the intention of excluding from FortiGuard web filters and web content filters
  • Block:
    • Block access
  • Allow:
    • Allowed and proceed to evaluation with subsequent web filter features (such as web content filters)
  • Monitor:
    • Same process as “permit” and logged

CLI settings

On the CLI config, the URL filter settings refer to the records in the URL filter config (config webfilter urlfilter) in the web filter profile config (config webfilter profile).

config webfilter profile
    edit "default"
        #omit
        config web
            #omit
            set urlfilter-table 1 "<-------------"
            #omit
        end
        #omit
    next
end

config webfilter urlfilter
    edit 1 "<-------------"
        set name "Auto-webfilter-urlfilter_9c7amcqzu"
        set comment ''
        set one-arm-ips-urlfilter disable
        set ip-addr-block disable
        config entries
            edit 1
                set url "*yahoo.co.jp"
                set type wildcard
                set action block
                set antiphish-action block
                set status enable
                set referrer-host ''
            next
        end
    next
end

Operation check

Suppose you have set the following URL filter:

If you access yahoo.co.jp with the Web filter profile including this URL filter applied to the policy, the following screen will be displayed and access will be blocked.

Check the URL filter log

You can see the URL filter logs on the GUI [Log & Report > Web Filter] screen.

Log display in CLI

You can check the local log in the CLI by following the steps below.

----
1. Log filter settings
----
FW01 # execute log filter category utm-webfilter

----
2. View log
----
FW01 # execute log display
5 logs found.
5 logs returned.

1: date=2021-10-30 time=14:05:47 eventtime=1635570347416888291 tz="+0900" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_l3j18n2e3" policyid=15 sessionid=800689 srcip=10.1.10.3 srcport=53855 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=183.79.94.59 dstport=995 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="pop.mail.yahoo.co.jp" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://pop.mail.yahoo.co.jp/" sentbyte=650 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"

2: date=2021-10-30 time=14:04:47 eventtime=1635570287443124397 tz="+0900" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_l3j18n2e3" policyid=15 sessionid=800568 srcip=10.1.10.3 srcport=53825 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=183.79.94.59 dstport=995 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="pop.mail.yahoo.co.jp" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://pop.mail.yahoo.co.jp/" sentbyte=650 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"

3: date=2021-10-30 time=14:04:21 eventtime=1635570261769893859 tz="+0900" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_l3j18n2e3" policyid=15 sessionid=800473 srcip=10.1.10.3 srcport=53807 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=182.22.25.124 dstport=80 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTP" hostname="yahoo.co.jp" profile="WebFilterProfile01" action="blocked" reqtype="referral" url="http://yahoo.co.jp/favicon.ico" referralurl="http://yahoo.co.jp/" sentbyte=430 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"

4: date=2021-10-30 time=14:04:21 eventtime=1635570261633373877 tz="+0900" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_l3j18n2e3" policyid=15 sessionid=800472 srcip=10.1.10.3 srcport=53806 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=182.22.25.124 dstport=80 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTP" hostname="yahoo.co.jp" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="http://yahoo.co.jp/" sentbyte=490 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"

5: date=2021-10-30 time=14:03:47 eventtime=1635570227420026337 tz="+0900" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_l3j18n2e3" policyid=15 sessionid=800362 srcip=10.1.10.3 srcport=53770 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=183.79.94.59 dstport=995 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="pop.mail.yahoo.co.jp" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://pop.mail.yahoo.co.jp/" sentbyte=650 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"

----
3. Log filter reset
----
FW01 # execute log filter reset

Blacklist and whitelist

There are two ways of thinking about web filtering: blacklist method and whitelist method.

  • Blacklist method:
    • In principle, allow access and block anything on a list called a blacklist.
  • Whitelist method:
    • In principle, block access and allow anything on a list called a whitelist.

FortiGate’s URL filters are blacklisted, allowing URLs that do not match any of the URL patterns specified in the URL filter.

If you want to set using the whitelist method

If you want to set the whitelist method to allow access only to the specified URL, add a rule to block all URLs at the end of the URL filter rule as shown in the image below, and for the URLs you want to allow above this rule Set the rules for.

Rule application order and change of order

As you might have guessed from the above explanation, the URL filter rules are applied in order from the top. Therefore, the order of application of rules is important in cases where permission and block rules are mixed.

Regarding how to change the rule application order, you can change the position (application order) by dragging and dropping each rule in the URL field in the rule list of the URL filter.

FortiGuard Web Filter

FortiGuard web filters control access by categorizing billions of web pages into different categories that users can allow or block.

When there is web traffic that is subject to the FortiGuard web filter, the access URL is sent to the nearest FortiGuard server and the URL category is returned.

FortiGate blocks or allows web access based on the categories and web filter profile settings returned by FortiGuard.

FortiGuard Web Filter Settings

FortiGuard web filters are configured with the [FortiGuard Category Based Filter] enabled in the web filter profile.

The category list is displayed when [ FortiGuard Category Based Filter] is enabled as shown below. You can see an example of the corresponding site by hovering the mouse pointer over the category name.

There are several sets of settings as Pre-configured filters that allow you to select the settings you want to base them on.

Action

  • Allow:
    • Allow access to sites in the category
  • Monitor:
    • Allow and log access to sites in the category
  • Block:
    • Block access to sites in the category
  • Warning:
    • Display a warning screen and browse the site if the user chooses to continue
  • Authenticate:
    • Require users to authenticate with FortiGate before granting access to sites in the category

Operation check

For blocks

Access bbc.co.uk by applying a web filter profile with the News and Media category Block.

Then the following screen will be displayed and access will be blocked.

For warning

Then change the filter action to Warning and go to bbc.co.uk again.

Then the following warning screen will be displayed.

The website is displayed when the user clicks [Proceed] on the warning screen above.

For authenticate

Next, let’s set the action of the filter to Authenticate.

When the action of the target category is authenticated on the setting screen, the following screen is displayed, so specify the user group you want to use for authentication. (Select from the user groups defined in FortiGate.)

After setting, access bbc.co.uk again.

Then, a screen similar to the warning screen will be displayed first as shown below, and when you click [Proceed], the authentication screen will be displayed.

Enter your credentials and click [Continue] to display the website.

FortiGuard Web Filter Log Check

You can see the FortiGuard Web Filter Logs on the GUI [Log & Report > Web Filter] screen.

Log display in CLI

You can check the local log in the CLI by following the steps below.

----
1. Log filter settings
----
FW01 # execute log filter category utm-webfilter

----
2. View log
----
FW01 # execute log display
157 logs found.
10 logs returned.

1: date=2021-10-30 time=14:35:45 eventtime=1635572145979475543 tz="+0900" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=15 sessionid=805728 srcip=10.1.10.3 srcport=56037 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=212.58.237.253 dstport=443 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="www.bbc.co.uk" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://www.bbc.co.uk/" sentbyte=517 rcvdbyte=0 direction="outgoing" msg="URL belongs to a category with warnings enabled" method="domain" cat=36 catdesc="News and Media"

2: date=2021-10-30 time=14:35:44 eventtime=1635572144917233501 tz="+0900" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=15 sessionid=805716 srcip=10.1.10.3 srcport=56026 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=212.58.237.253 dstport=443 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="www.bbc.co.uk" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://www.bbc.co.uk/" sentbyte=517 rcvdbyte=0 direction="outgoing" msg="URL belongs to a category with warnings enabled" method="domain" cat=36 catdesc="News and Media"

3: date=2021-10-30 time=14:35:44 eventtime=1635572144859426517 tz="+0900" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=15 sessionid=805713 srcip=10.1.10.3 srcport=56023 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=212.58.237.253 dstport=443 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="www.bbc.co.uk" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://www.bbc.co.uk/" sentbyte=728 rcvdbyte=0 direction="outgoing" msg="URL belongs to a category with warnings enabled" method="domain" cat=36 catdesc="News and Media"

4: date=2021-10-30 time=14:35:43 eventtime=1635572143641279701 tz="+0900" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=15 sessionid=805701 srcip=10.1.10.3 srcport=56013 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=212.58.237.253 dstport=443 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="www.bbc.co.uk" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://www.bbc.co.uk/" sentbyte=728 rcvdbyte=0 direction="outgoing" msg="URL belongs to a category with warnings enabled" method="domain" cat=36 catdesc="News and Media"

5: date=2021-10-30 time=14:35:32 eventtime=1635572131988261702 tz="+0900" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=15 sessionid=805589 srcip=10.1.10.3 srcport=55917 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=151.101.192.81 dstport=80 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTP" hostname="bbc.co.uk" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="http://bbc.co.uk/" sentbyte=256 rcvdbyte=0 direction="outgoing" msg="URL belongs to a category with warnings enabled" method="domain" cat=36 catdesc="News and Media"

#omit

----
3. Log filter reset
----
FW01 # execute log filter reset

Web content filter

Web content filters control access to web content by blocking web pages that contain specific words or patterns.

Content rating

  • You can specify prohibited words and phrases and add a number (or score) to the importance of those words and phrases.
  • Web Content Filter When scanning banned content is detected, the score of the banned word or phrase found on that page is added.
  • FortiGate blocks the page if the total score is higher than the threshold set in his web filter profile
  • The web content filter has a default score of 10 and a default threshold of 10. This means that by default, web pages are blocked by a single match.
  • These settings can only be set in the CLI

Web content filter settings

Web content filters are set in the Web filter profile under [Static URL Filters > Content Filter].

Click [Create New] at the top of the table in the content filter column to display the new web content filter setting screen, and make any settings.

In the image below, it is set to block pages including “Casino”.

CLI settings

On the CLI config, the web content filter settings refer to the records in the web content filter config (config webfilter content) in the web filter profile config (config webfilter profile).

config webfilter profile
    edit "default"
        #omit
        config web
            set bword-table 1 "<-------------"
        end
        #omit
    next
end

config webfilter content
    edit 1 "<-------------"
        set name "Auto-webfilter-content_se02e0yjy"
        config entries
            edit "*Casino*"
                set status enable
            next
        end
    next
end

Operation check

Specify a specific pattern to set the action to block.

After that, when you access a website that contains a character string that corresponds to that pattern, the following screen will be displayed and access will be blocked.

For HTTPS communication

In the case of HTTPS communication, content filtering is not possible because the communication is encrypted. If you also want to filter HTTPS, you probably need to use SSL inspection as well.

Check the log of the Web Content Filter

You can see the Web Content Filter Logs on the GUI [Log & Report > Web Filter] screen.

Check web filter statistics

You can check the statistics with the diagnose webfilter stats list root command.

FW01 # diagnose webfilter stats list root
Proxy/flow URL filter stats:
 request:   0
 blocked:   0
 allowed:   0
 overridden:0
 logged:    4
 pending:   0

References

Web filter | Administration Guide
docs.fortinet.com
webfilter | CLI Reference
docs.fortinet.com
FortiGate Mastery Web Book
Getting Started with FortiGate How to access the GUI of factory default FortiGate Basic knowledge about configuration co...
network-knowledge.work
2021.11.01
スポンサーリンク

Comments

Copied title and URL