Sign Up
  1. Znode
  2. Solution home
  3. Developer Documentation - V9
  4. Getting Started with Znode - V9

Cloudflare Integration With Znode

TABLE OF CONTENTS

Introduction

Znode partnered with Cloudflare to provide an out-of-box integration that can enable advanced security from bot attacks and also provide a boost in site performance via their global edge network (caching). The integration provides the following essential features:

  1. Web Application Firewall (WAF)
  2. Comprehensive DDoS Protection
  3. Bot Management
  4. Edge Computing - Largest Global Network

The Znode application will manage the following points of Cloudflare

  1. The Cloudflare cache purging can be done from the Znode application in 2 ways
  2. Manual Full Store Cache purge

    1. Admin users can manually purge the Cloudflare cache for the full store, See #4 here.

  3. Auto Purge Specific URL Cache
  4. The specific cache will be auto-purged from Cloudflare when you will publish the below-mentioned elements:
    1. Store
    2. Catalog
    3. Category
    4. Product and 
    5. CMS

Assumption 

  1. Users should have a business account on Cloudflare.
  2. This document doesn’t cover detailed Cloudflare knowledge and assumes that the administrator has knowledge of Cloudflare administration.
  3. The Znode application should be hosted on a public IP/domain so that Cloudflare can access the Znode application.

How to use the Cloudflare caching feature with Znode

By doing the following steps Cloudflare will cache the Znode webstore application

Setup needs to be done in the Cloudflare portal

  1. Add the website URLs in the DNS tab with the Name and IPv4 address. This setup will host the website on Cloudflare.Note - If the website is already hosted then this step can be skipped
    Note- If the website is already hosted then this step can be skipped
    1. Rule 1 - Bypass GetCartCount Ajax
      1. URL - {domain name}/Home/GetCartCount
      2. Setting - Cache Level: Bypass
    2. Rule 2 - Bypass Login Ajax
      1. URL - {webstore URL}/User/Login*
      2. Setting - Cache Level: Bypass
    3. Rule 3 - Bypass Cart
      1. URL - {webstore URL}/cart*
      2. Setting - Cache Level: Bypass
    4. Rule 5 - Bypass recent view products
      1. URL - {webstore URL}/Product/GetRecentViewProducts*
      2. Setting - Cache Level: Bypass
    5. Rule 6 -  Bypass the checkout page
      1. URL - {webstore URL}/checkout/index*
      2. Setting - Cache Level: Bypass
    6. Rule 7 -  Bypass order receipt
      1. URL - {webstore URL}/Checkout/OrderCheckoutReceipt*
      2. Setting - Cache Level: Bypass
    7. Rule 8 -  Bypass the wishlist page
      1. URL - {webstore URL}/User/Wishlist*
      2. Setting - Cache Level: Bypass
    8. Rule 9 -  Bypass the user history page
      1. URL - {webstore URL}/User/History*
      2. Setting - Cache Level: Bypass
    9. Rule 10 -  Bypass user address URL
      1. URL - {webstore URL}/User/AddressBook*
      2. Setting - Cache Level: Bypass
    10. Rule 11 -  Bypass user review URL
      1. URL - {webstore URL}/User/Reviews*
      2. Setting - Cache Level: Bypass
    11. Rule 12 -  Bypass gift card history URL
      1. URL - {webstore URL}/User/GiftCardHistory*
      2. Setting - Cache Level: Bypass
    12. Rule 13 -  Bypass profile page
      1. URL - {webstore URL}/User/EditProfile*
      2. Setting - Cache Level: Bypass
    13. Rule 14 -  Bypass saved credit card URL
      1. URL - {webstore URL}/User/GetSavedCreditCards*
      2. Setting - Cache Level: Bypass
    14. Rule 15 -  Bypass affiliate information page
      1. URL - {webstore URL}/User/AffiliateInformation*
      2. Setting - Cache Level: Bypass
    15. Rule 16 - Bypass GetPriceWithInventory Ajax call
      1. URL - {webstore URL}/Product/GetPriceWithInventory*
      2. Setting - Cache Level: Bypass
    16. Rule 17 - Bypass LoginStatus Ajax call
      1. URL - {webstore URL}/User/LoginStatus*
      2. Setting - Cache Level: Bypass
    17. Follow the same steps to add additional URLs which you would like to serve dynamically without cache. The same goes for any custom Page or Ajax request that you would like to bypass.
  2. Add the following rules in the “Page Rule” tab of the Cloudflare account to bypass non-cacheable or dynamic content URLs (which needs real-time values rather than cached content)
  3. Add the following rules in the “Page Rule” tab of the Cloudflare account to cache everything. This rule should be added to the last of the rules list. 
    1. Cache everything on the webstore URL
      1. URL - {webstore URL}/*
      2. Setting - Cache Level: Cache Everything, Edge Cache TTL: 12 hours
      3. Order - Last
        Note - In-depth documentation about setting up page rules for a website can be found on the Cloudflare Official page - https://support.cloudflare.com/hc/en-us/articles/218411427-Understanding-and-Configuring-Cloudflare-Page-Rules-Page-Rules-Tutorial-#overview
  4. Users can prevent web scraping/High precision DOS attacks. The same has been described in the link https://www.cloudflare.com/learning/bots/what-is-rate-limiting/
  5. Cloudflare can restrict URL access to trusted IP ranges using Zone Lockdown. https://support.cloudflare.com/hc/en-us/articles/115001595131-Understanding-Cloudflare-Zone-Lockdown
    Sample instruction: 
    • Ask customers for which regions; Mexico, CA, and UK.
    • Ask customers for which 3rd party apps are going to need access to the API.
  6. In Cloudflare, different firewall rules can be set for e.g. Blocking production site access for countries or continents https://developers.cloudflare.com/firewall/cf-firewall-rules/
  7. Important: By default, Cloudflare caches the images. Znode serves images from the API domain, so there is no need for any special rule to cache images coming from APIs (if the API domain is already proxied through Cloudflare). In fact, this was observed that, if a rule is put to bypass the API domain from the cache it would negatively affect the caching of images as well.

Setup needs to do in the Znode API Config file

There are a few keys related to Cloudflare, which need to be set up in API’s web. config file:

In API’s web.config file, Under “CloudflareConfigSection” change the value of “CloudflareApiKey”. Where to get the value of this key in Cloudflare?

Setup needs to do in the Znode admin application

  1. Enable the Cloudflare from the following menu
    1. Store -> Additional Attributes -> Cloudflare Setting
  2. Add zone id provided by Cloudflare account. The zone id can be found in the overview tab of the Cloudflare account.

Purging of Cloudflare cache 

From Znode application

  1. The Cloudflare purging from the Znode admin application can only work if the Enable Cloudflare flag is enabled for the store.
  2. When an admin user will publish the Store, Catalog, and CMS content from the Znode admin application then all the Cloudflare cache will be cleared of all the websites which are associated with provided zoneId to the store.
  3. When an admin user will publish the category and product from the Znode admin application then the cache of the URL associated with the category or product will be purged.
  4. Cloudflare caching can be purged manually from Admin -> Global Setting -> Cache management. Here users can select the domain for which the Cloudflare cache needs to be purged.




From Cloudflare account

Purging of the Cloudflare cache can be done from the Cloudflare account in two ways

  1. Purge everything - All Cloudflare cache can be cleared from the Purge everything button in the caching tab of the Cloudflare account.
  2. Custom Purge - Custom purge provides three options to purge the Cloudflare cache
    1. Purge by URL - Any cache related to a particular URL will be purged from the Cloudflare cache.
    2. Follow this link for some Cloudflare enterprise account features around cache purging. https://support.cloudflare.com/hc/en-us/articles/200169246-How-do-I-purge-my-cache-#2Cmoyau5BdK0sEYFlpoxq8

Common Issues and their workaround:

Znode Webstore and Admin Performance issue

  1. Issue:
    When the API is added to the project's DNS entry in CloudFlare (As Proxied) (Screenshot below) to serve the DAM images through CF CDN, it starts to impact the Webstore and Admin app performance.
     
  2. Root Cause:
    The Webstore and Admin app perform slow, because, to reach the API server, both the apps will have to go through the CF proxy server over an internet connection. (See diagram below)
     
  3. Workaround
    1. Preferred option:
      This option provides the best performance.“vhost” file can be configured on the Windows machine of the Webstore/Admin server, which helps in DNS lookup. If the API is on the local machine, an entry can be made such that the API domain resolves to localhost. If the API is not on the local machine, an entry can be made such that the API domain resolves to the IP of the machine hosting the API, or, to the IP of the load balancer in front of the API nodes if applicable.
      In this “vhost” file, the API “domain” can be mapped with a private network “IP” (See diagram below)
       
    2. Alternative option:
      This option increases latency and should be avoided, but is noted here for the sake of documentation. Keeping the CF Proxy server and Znode server in the closest possible proximity to have minimal latency can be tried. In this approach, the hosting server doesn’t require any additional configuration/handling. Which will be the desired option in multi-node (LB) deployment. But still, it will have the “internet” network cost, and won’t have the most optimum performance. And also, CF CDN is an edge service, so it can’t really be achieved always.

Znode Publish or Search Index creation process:

Note: This is also related to the above issue #1, but specifically occurs with publishing and with any long-running HTTP Requests while working with CloudFlare.

  1. Issue
    It has been observed that when Znode API is proxied (DNS configuration) through Cloudflare, the Znode Search Index process was failing for a large (time-consuming) catalog. It used to timeout the connection. And in Znode, the “Index failed” message is seen as there were not ample logs in the TaskScheduler>Indexer. cs class.
  2. Root Cause
    Below is the “Publish / Create Index” request flow, with an explanation:

    The call between TaskScheduler and Indexing API was connected through CloudFlare proxy. And as it is known, the Indexing process may take up to many minutes based on catalog size/network, and Cloudflare will not respect the "timeout" setting that is applied on WebRequest in TaskScheduler>Indexer. cs class and it terminates the request if it takes more than 100 seconds. Which will result in Index process failure.
    Upon doing some more debugging and logs, it was seen that the TaskScheduler was receiving the “Connection Closed” error with Error 524 code.
    This “Error 524” code is Cloudflare-specific code, and after searching for it, it was found that in CloudFlare, “Any HTTP request that doesn't receive a response from the origin server for over 100 seconds (or over 600 seconds for enterprise customers) will be timed out and you'll see the 524 A Timeout Occurred error.”
  3. Workaround 
    To avoid TaskScheduler to connect to Znode APIs via CloudFlare proxy, API's DNS lookup was added on the local Virtual Host (Vhost file) file in windows.
    The API domain was added with the corresponding Private IP in the Vhost file. So now, DNS for API will be looked up locally, and it will connect to API on the private network, rather than routing through Cloudflare.

Mapping users in the origin server while using Cloudflare

  1. Issue: 
    When requests are coming from Cloudflare to origin servers, the standard “client IP” header will have CloudFlare’s IP address but the actual user’s IP. And in some cases, Application or for debugging purposes, actual User IPs may be required.
  2. Solution:
    Typically any Gateway (WAF, CDN, Proxy servers) passes the original IP of the user to the origin server in some different/custom header (HTTP Request header). Similarly, CloudFlare also passes the user’s IP in the following custom header:
    • “x-forwarded-for” OR “cf-connecting-ip” [VERIFIED in IIS]: This HTTP Request header is supported with the business plan of CF. Link
      To log the user’s IP in IIS logs, one can add the “Custom Field” with “x-forwarded-for” OR “cf-connecting-ip” header in the Log settings of the IIS website. Steps from the CF website:
      Also, if the Application code needs a user's IP, they will have to access it from the asp.net’s HttpWebRequest object, under the WebHeaderCollection object.

.Net C# code E.g: (Sample only, modify it as per the context and need)

References:

https://support.cloudflare.com/hc/en-us/articles/200170786-Restoring-original-visitor-IPs

Limitations

  1. Only web stores are compatible to work with Cloudflare caching and page rules defined above.
  2. Cloudflare does not provide any facility which allows caching for a particular section on the web page.
  3. Real-time ajax request URLs need to be specified under Cloudflare page rules to exclude them from caching. This takes some time to get the perfect setup for any site.

Important

  1. Cloudflare is enabled for Webstore only.
  2. Real-time data of price and inventory will be displayed on PDP using Ajax calls and data on PLP will always come from the Cloudflare cache.
  3. After purging the Cloudflare cache, users will have to wait for 30 sec to see an impact on the web store.
  4. If the setup of caching has been done in the Cloudflare account but not enabled from the Znode admin application then Cloudflare cache purging will not work from the Znode application.


Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.

Getting Started with Znode - V9