How can I tell if the errors I'm seeing in Kibana are coming from my origin? Follow
STARTING FROM THE SAME PAGE
This article assumes that you have already navigated to your environment's Kibana console. The image below should give you an idea of what that should look like.
GATHERING THE CORRECT DATA
- The first thing that needs to be determined is if any status codes between 500 and 599 are being logged in Kibana. To do this you will need to type "status:[500 TO 599]" into the Kibana search bar (see image above for an example). If you are not aware of what the various status codes mean, take a look at this Wikipedia article on HTTP status codes and take a look at 5xx errors specifically.
- After determining that there are 5xx errors being delivered, you will need to determine if the Egress Proxy has generated any 5xx last_proxy-access-logs. The easiest way to do this is to add: "AND _type:last_proxy-access-log" to your search filter. The result should look like the following.NOTES:
- The Section Egress Proxy is responsible for proxying requests flowing through the Section platform to your origin and receiving responses from your origin.
- At this point, you should see the number of results drop substantially due to the fact that every request that flows through Section will generate multiple logs and you have now just filtered down to one log per request.
- Next, you will need to add a few pertinent fields from your search results to the results list. You will want to add "upstream_status", "upstream_response_time_seconds", and "upstream_bytes_received". The easiest way to add these columns is to navigate the left-hand column of "Available Fields" and select the "add" button. See the image below for details. Note: The fields are listed alphabetically, except for commonly used fields, which are listed at the top first. So if you don't see a field you are looking for, scroll up to the top of the list and look for it there.
MAKING THE CALL
The image below is similar to what you should now be seeing in Kibana.
You now have the information needed in front of you to diagnose if the 5xx errors in Kibana are originating from your origin. Reading the data and making the call can still be difficult at times, but in general, if you see values (other than 0) in the "upstream_status", "upstream_response_time_seconds", and "upstream_bytes_received" fields in your search results, then the errors are originating from your origin (see image above). There are many variations of results that you could see at this point which we will not cover here, but another common scenario is that the "upstream_status" and "upstream_response_time_seconds" have values, but the "upstream_bytes_received" have only 0s. This means that the Section Egress proxy was unable to establish a TCP connection with your origin servers.