Work environment
- FortiGate 60E
- version 7.0.1
Web filter function overview
The Web Filter feature allows you to apply a Web Filter profile to a (firewall) policy to limit or control user access to Web resources for traffic that matches that policy according to the contents of the applied profile.
Three major components
- Web content filter
- Block web pages that contain the specified word or pattern
- URL filter
- Use URLs and URL patterns to block or allow web access to matching URLs, or block malicious URLs detected by FortiSandbox.
- FortiGuard Web Filtering Service
- Provides many additional categories that can be used to filter web traffic, enabling categorical access control
Default web filter profile
By default, there are three web filter profiles:
- default
- monitor-all
- wifi-default
You can customize and use these profiles, or you can create and use new original profiles.
Web filter profile setting screen
If you set it from the GUI, you can do it on the [Security Profiles > Web Filter] screen.
To create a new profile, click the [Create New] button.
Flow-based and Proxy-based
You can select flow-based or proxy-based as the value of [Feature set], which is a setting item of the profile.
There are some features that can only be used with proxy-based, but proxy-based is said to have lower throughput than flow-based.
◆Setting items for flow-based
◆Setting items for proxy-based
The setting items marked with (P)
in the image above are functions that can be used only when proxy-based.
Config in CLI
The config of the web filter profile in the CLI is config webfilter profile
.
config webfilter profile
edit "default"
set comment "Default web filtering."
set feature-set proxy
config ftgd-wf
unset options
config filters
edit 1
set category 2
set action block
next
edit 2
set category 7
set action block
next
edit 3
set category 8
set action block
next
...
edit 23
set category 91
set action block
next
end
end
next
end
Web filter function application order
The order in which it is applied for multiple web filter functions is as follows.
- URL filter
- FortiGuard Web Filter
- Web content filter
- Web script filter
- Antivirus scan
By using this application order, you can design the URL filter to exclude only the specified URL from the subsequent filtering.
URL filter settings
URL filters allow you to block, allow, monitor, and so on specific URLs that have a pattern that contains the specified text or regular expression.
URL filters are set in the [Static URL Filter > URL Filter] field on the Web filter profile setting screen.
URL filter type
- Simple:
- Judge as a hit if it exactly matches the specified URL
- Regular Expression/Wildcard:
- Determine based on regular expression or wildcard rules
URL filter action
- Exempt:
- Exempted from evaluation by subsequent Web filter functions (Web content filter, etc.) (permitted)
- Used with the intention of excluding from FortiGuard web filters and web content filters
- Block:
- Block access
- Allow:
- Allowed and proceed to evaluation with subsequent web filter features (such as web content filters)
- Monitor:
- Same process as “permit” and logged
CLI settings
On the CLI config, the URL filter settings refer to the records in the URL filter config (config webfilter urlfilter
) in the web filter profile config (config webfilter profile
).
config webfilter profile
edit "default"
#omit
config web
#omit
set urlfilter-table 1 "<-------------"
#omit
end
#omit
next
end
config webfilter urlfilter
edit 1 "<-------------"
set name "Auto-webfilter-urlfilter_9c7amcqzu"
set comment ''
set one-arm-ips-urlfilter disable
set ip-addr-block disable
config entries
edit 1
set url "*yahoo.co.jp"
set type wildcard
set action block
set antiphish-action block
set status enable
set referrer-host ''
next
end
next
end
Operation check
Suppose you have set the following URL filter:
If you access yahoo.co.jp
with the Web filter profile including this URL filter applied to the policy, the following screen will be displayed and access will be blocked.
Check the URL filter log
You can see the URL filter logs on the GUI [Log & Report > Web Filter] screen.
Log display in CLI
You can check the local log in the CLI by following the steps below.
----
1. Log filter settings
----
FW01 # execute log filter category utm-webfilter
----
2. View log
----
FW01 # execute log display
5 logs found.
5 logs returned.
1: date=2021-10-30 time=14:05:47 eventtime=1635570347416888291 tz="+0900" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_l3j18n2e3" policyid=15 sessionid=800689 srcip=10.1.10.3 srcport=53855 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=183.79.94.59 dstport=995 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="pop.mail.yahoo.co.jp" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://pop.mail.yahoo.co.jp/" sentbyte=650 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"
2: date=2021-10-30 time=14:04:47 eventtime=1635570287443124397 tz="+0900" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_l3j18n2e3" policyid=15 sessionid=800568 srcip=10.1.10.3 srcport=53825 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=183.79.94.59 dstport=995 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="pop.mail.yahoo.co.jp" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://pop.mail.yahoo.co.jp/" sentbyte=650 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"
3: date=2021-10-30 time=14:04:21 eventtime=1635570261769893859 tz="+0900" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_l3j18n2e3" policyid=15 sessionid=800473 srcip=10.1.10.3 srcport=53807 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=182.22.25.124 dstport=80 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTP" hostname="yahoo.co.jp" profile="WebFilterProfile01" action="blocked" reqtype="referral" url="http://yahoo.co.jp/favicon.ico" referralurl="http://yahoo.co.jp/" sentbyte=430 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"
4: date=2021-10-30 time=14:04:21 eventtime=1635570261633373877 tz="+0900" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_l3j18n2e3" policyid=15 sessionid=800472 srcip=10.1.10.3 srcport=53806 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=182.22.25.124 dstport=80 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTP" hostname="yahoo.co.jp" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="http://yahoo.co.jp/" sentbyte=490 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"
5: date=2021-10-30 time=14:03:47 eventtime=1635570227420026337 tz="+0900" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_l3j18n2e3" policyid=15 sessionid=800362 srcip=10.1.10.3 srcport=53770 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=183.79.94.59 dstport=995 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="pop.mail.yahoo.co.jp" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://pop.mail.yahoo.co.jp/" sentbyte=650 rcvdbyte=0 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"
----
3. Log filter reset
----
FW01 # execute log filter reset
Blacklist and whitelist
There are two ways of thinking about web filtering: blacklist method and whitelist method.
- Blacklist method:
- In principle, allow access and block anything on a list called a blacklist.
- Whitelist method:
- In principle, block access and allow anything on a list called a whitelist.
FortiGate’s URL filters are blacklisted, allowing URLs that do not match any of the URL patterns specified in the URL filter.
If you want to set using the whitelist method
If you want to set the whitelist method to allow access only to the specified URL, add a rule to block all URLs at the end of the URL filter rule as shown in the image below, and for the URLs you want to allow above this rule Set the rules for.
Rule application order and change of order
As you might have guessed from the above explanation, the URL filter rules are applied in order from the top. Therefore, the order of application of rules is important in cases where permission and block rules are mixed.
Regarding how to change the rule application order, you can change the position (application order) by dragging and dropping each rule in the URL field in the rule list of the URL filter.
FortiGuard Web Filter
FortiGuard web filters control access by categorizing billions of web pages into different categories that users can allow or block.
When there is web traffic that is subject to the FortiGuard web filter, the access URL is sent to the nearest FortiGuard server and the URL category is returned.
FortiGate blocks or allows web access based on the categories and web filter profile settings returned by FortiGuard.
FortiGuard Web Filter Settings
FortiGuard web filters are configured with the [FortiGuard Category Based Filter] enabled in the web filter profile.
The category list is displayed when [ FortiGuard Category Based Filter] is enabled as shown below. You can see an example of the corresponding site by hovering the mouse pointer over the category name.
There are several sets of settings as Pre-configured filters that allow you to select the settings you want to base them on.
Action
- Allow:
- Allow access to sites in the category
- Monitor:
- Allow and log access to sites in the category
- Block:
- Block access to sites in the category
- Warning:
- Display a warning screen and browse the site if the user chooses to continue
- Authenticate:
- Require users to authenticate with FortiGate before granting access to sites in the category
Operation check
For blocks
Access bbc.co.uk
by applying a web filter profile with the News and Media category Block
.
Then the following screen will be displayed and access will be blocked.

For warning
Then change the filter action to Warning
and go to bbc.co.uk
again.
Then the following warning screen will be displayed.

The website is displayed when the user clicks [Proceed] on the warning screen above.
For authenticate
Next, let’s set the action of the filter to Authenticate
.
When the action of the target category is authenticated on the setting screen, the following screen is displayed, so specify the user group you want to use for authentication. (Select from the user groups defined in FortiGate.)

After setting, access bbc.co.uk
again.
Then, a screen similar to the warning screen will be displayed first as shown below, and when you click [Proceed], the authentication screen will be displayed.
Enter your credentials and click [Continue] to display the website.
FortiGuard Web Filter Log Check
You can see the FortiGuard Web Filter Logs on the GUI [Log & Report > Web Filter] screen.
Log display in CLI
You can check the local log in the CLI by following the steps below.
----
1. Log filter settings
----
FW01 # execute log filter category utm-webfilter
----
2. View log
----
FW01 # execute log display
157 logs found.
10 logs returned.
1: date=2021-10-30 time=14:35:45 eventtime=1635572145979475543 tz="+0900" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=15 sessionid=805728 srcip=10.1.10.3 srcport=56037 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=212.58.237.253 dstport=443 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="www.bbc.co.uk" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://www.bbc.co.uk/" sentbyte=517 rcvdbyte=0 direction="outgoing" msg="URL belongs to a category with warnings enabled" method="domain" cat=36 catdesc="News and Media"
2: date=2021-10-30 time=14:35:44 eventtime=1635572144917233501 tz="+0900" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=15 sessionid=805716 srcip=10.1.10.3 srcport=56026 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=212.58.237.253 dstport=443 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="www.bbc.co.uk" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://www.bbc.co.uk/" sentbyte=517 rcvdbyte=0 direction="outgoing" msg="URL belongs to a category with warnings enabled" method="domain" cat=36 catdesc="News and Media"
3: date=2021-10-30 time=14:35:44 eventtime=1635572144859426517 tz="+0900" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=15 sessionid=805713 srcip=10.1.10.3 srcport=56023 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=212.58.237.253 dstport=443 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="www.bbc.co.uk" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://www.bbc.co.uk/" sentbyte=728 rcvdbyte=0 direction="outgoing" msg="URL belongs to a category with warnings enabled" method="domain" cat=36 catdesc="News and Media"
4: date=2021-10-30 time=14:35:43 eventtime=1635572143641279701 tz="+0900" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=15 sessionid=805701 srcip=10.1.10.3 srcport=56013 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=212.58.237.253 dstport=443 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTPS" hostname="www.bbc.co.uk" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="https://www.bbc.co.uk/" sentbyte=728 rcvdbyte=0 direction="outgoing" msg="URL belongs to a category with warnings enabled" method="domain" cat=36 catdesc="News and Media"
5: date=2021-10-30 time=14:35:32 eventtime=1635572131988261702 tz="+0900" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=15 sessionid=805589 srcip=10.1.10.3 srcport=55917 srcintf="VLAN10" srcintfrole="lan" srcuuid="769878e8-1967-51ea-d211-302593b302ea" dstip=151.101.192.81 dstport=80 dstintf="wan1" dstintfrole="wan" dstuuid="da05e890-05d8-51ea-f01b-cfba759f8e4b" proto=6 service="HTTP" hostname="bbc.co.uk" profile="WebFilterProfile01" action="blocked" reqtype="direct" url="http://bbc.co.uk/" sentbyte=256 rcvdbyte=0 direction="outgoing" msg="URL belongs to a category with warnings enabled" method="domain" cat=36 catdesc="News and Media"
#omit
----
3. Log filter reset
----
FW01 # execute log filter reset
Web content filter
Web content filters control access to web content by blocking web pages that contain specific words or patterns.
Content rating
- You can specify prohibited words and phrases and add a number (or score) to the importance of those words and phrases.
- Web Content Filter When scanning banned content is detected, the score of the banned word or phrase found on that page is added.
- FortiGate blocks the page if the total score is higher than the threshold set in his web filter profile
- The web content filter has a default score of 10 and a default threshold of 10. This means that by default, web pages are blocked by a single match.
- These settings can only be set in the CLI
Web content filter settings
Web content filters are set in the Web filter profile under [Static URL Filters > Content Filter].
Click [Create New] at the top of the table in the content filter column to display the new web content filter setting screen, and make any settings.
In the image below, it is set to block pages including “Casino”.
CLI settings
On the CLI config, the web content filter settings refer to the records in the web content filter config (config webfilter content
) in the web filter profile config (config webfilter profile
).
config webfilter profile
edit "default"
#omit
config web
set bword-table 1 "<-------------"
end
#omit
next
end
config webfilter content
edit 1 "<-------------"
set name "Auto-webfilter-content_se02e0yjy"
config entries
edit "*Casino*"
set status enable
next
end
next
end
Operation check
Specify a specific pattern to set the action to block
.
After that, when you access a website that contains a character string that corresponds to that pattern, the following screen will be displayed and access will be blocked.
For HTTPS communication
In the case of HTTPS communication, content filtering is not possible because the communication is encrypted. If you also want to filter HTTPS, you probably need to use SSL inspection as well.
Check the log of the Web Content Filter
You can see the Web Content Filter Logs on the GUI [Log & Report > Web Filter] screen.
Check web filter statistics
You can check the statistics with the diagnose webfilter stats list root
command.
FW01 # diagnose webfilter stats list root
Proxy/flow URL filter stats:
request: 0
blocked: 0
allowed: 0
overridden:0
logged: 4
pending: 0
Comments