IP Intelligence - Proxy / VPN / Bad IP Detection

IP Intelligence is a service that determines how likely an IP address is a proxy / VPN / bad IP using advanced mathematical and modern computing techniques

Detect bot, proxy, and VPN traffic to:
  • Greatly reduce fraud on e-commerce sites (anti-fraud)
  • Protect your site from automated hacking attempts such as XSS, SQLi, brute force attacks, application scanning and many others
  • Protect your site from crawlers that steal your content
  • Prevent users from abusing promotional offers / multiple sign-ups / affiliate abuse
  • Stop bots from scraping your content or bots spamming your website
  • Serve traffic / content to real users, not bots. Reduce fake views, clicks, and activity that results in click fraud and view fraud (anti-bot detection)
  • Prevent trolls / people that are trying to bypass a ban
  • Adjust your system to limit access (such as not allowing them to change their password, their email, etc) to prevent account hijacking
  • Since the system returns a real value and there's different flag options, you can customize the level of protection for a particular time frame and adjust accordingly
  • Use it with a combination of another fraud prevention service to make it even better. Some fraud prevention services do not explicitly look for proxy / VPN / bad IPs
The system is serving millions of API requests a week and growing as more people find it useful in protecting their online infrastructure. Our service is used by gaming communities, e-commerce websites, research universities & institutions, law enforcement, and large financial institutions. Not all proxy / VPN detection services are the same. The techniques involved can be vastly different and produce noticeable differences. Feel free to compare the results from this service to any other, including paid options from various vendors.
It is recommended that you thoroughly read the information below before implementation.

Assumptions

The following assumptions must be met for the sake of accuracy and correctness.
  • It is assumed that the IP you're looking up is making a request to your services on an application level. If you block IPs on a lower level, important services such as DNS maybe be blocked which is not desired. Be sure the source IP addresses are correct (not spoofed) if you're trying to protect a UDP based service.
  • If your online services involve multiple servers or external services that interact with your online infrastructure, it is assumed that you do not look up these IPs or the IPs are whitelisted on your system.
  • A valid email that is checked frequently must be used in the contact field or else your service might be disabled without notice because there is no way to contact you.
  • If you are using the API interface, please do not exceed more than 500 queries per day & 15 queries per minute. Custom packages are available if you contact me. More information is available in the FAQs.
  • If you believe the results are incorrect, please contact me so I can look into it. I will happily correct any issues.
  • By using this service, you agree to the Terms of Service listed below.

API

Expected Input
The proxy check system takes in an input via HTTP GET request. The URL is http://check.getipintel.net/check.php and the parameter is ip . The system fully supports IPv4 with partial support for IPv6.

Include Your Contact Information
Include your contact information so I can notify you if a problem arise or if there are core changes. In some situations, people query the system in a wrong manner and assume everything is working (but due to the lack of or improper handling of error codes), it's not the case. Since I only have the connecting IP address, I cannot help the person correct the error.
To include your contact information, add another parameter to your request called contact and provide your email.

A typical query looks like:
http://check.getipintel.net/check.php?ip=IPHere&contact=YourEmailAddressHere


Do not use URL encoding on the input parameters.

All queries that do not contain accurate contact information will be rejected with an error or it'll be dropped by the firewall.

Start with flags=m option if only proxy / VPN detection is needed. If flags=m does not have a noticeable impact, then use flags=b. The default query (no flags) is mostly used infront of a payment gateway to protect against fraud because bad IP detection is included.
If you are contacted, please respond in 2 days or the contact information could be considered as inaccurate. Your information will only be used for the purpose of communication with GetIPIntel.




Expected Output
On a valid request, the system will return a value between 0 - 1 (inclusive) of how likely the given IP is a proxy. On error, a negative value will be returned. If format=json is used, a valid JSON format will be returned with extra information, see below for details.





Interpretation of the Results
If a value of 0.50 is returned, then it is as good as flipping a 2 sided fair coin, which implies it's not very accurate. From my personal experience, values > 0.95 should be looked at and values > 0.99 are most likely proxies. Anything below the value of 0.90 is considered as "low risk". Since a real value is returned, different levels of protection can be implemented. It is best for a system admin to test some sample datasets with this system and adjust implementation accordingly. I only recommend automated action on high values ( > 0.99 or even > 0.995 ) but it's always best to manually review IPs that return high values. For example, mark an order as "under manual review" and don't automatically provision the product for high proxy values. Be sure to experiment with the results of this system before you use it live on your projects. If you believe the result is wrong, don't hesitate to contact me, I can tell you why. If it's an error on my end, I'll correct it. If you email me, expect a reply within 12 hours.




Optional settings
  • flags=m is used when you're only looking for the value of "1" as the result. The m flag skips the dynamic checks and only uses dynamic ban lists. See Variations of Implementation and What are dynamic checks? for detailed explanation.
  • flags=b is used when you want to use dynamic ban and dynamic checks with partial bad IP check. See Variations of Implementation for detailed explanation.
  • flags=f is used when you want to force the system to do a full lookup, which can take up to 5 seconds. See Variations of Implementation for detailed explanation.
  • flags=n is used to exclude real time block list. Append the character "n" if you're already using flags=m, b, or f. For example, flags=nm.
  • oflags=b is used when you want to see if the IP is considered as bad IP. Note that when using flags option, this result can vary due to the included datasets. Please see the comparsion table for more information.
  • oflags=c is used when you want to see which country the IP came from / which country the IP belongs to (GeoIP Location). Currently in alpha testing.
  • oflags=i is used when you want to exclude iCloud Relay Egress IPs, Google Cloud One VPN, or some other similiar service. They are by definition a proxy/VPN IP, however, having this additional data may help you make a more informed decision.
  • oflags=a is used when you want to see the ASN number of the IP.
  • format=json returns the result in JSON format with extra information.




Variations of Implementation

flags=m
Use Dynamic Ban List Only (Skip Dynamic Check and Bad IP Checks)
If you get a value between 0 - 1, exclusive (like 0.99, 0.99999, 0.97), these values are generated by dynamic checks which looks for characteristics of the given IP. IPs that are either manually banned or seen on a public proxy site will return a value of 1. If you only want manually banned or public proxies, then in your code just look for the value "1". However, there are many IPs that haven't gone through manual review and IPs can change behavior very frequently (which is why dynamic checks exist in the first place). If you only look for the value of "1", then expect to have more proxy / VPN / bad IPs go through your system, however, false positives are less likely if you use the dynamic ban list option.

If you wish to use only manually banned & public proxy IPs, append the parameter &flags=m, the system will only return a result of 0 or 1. This option is the best to start off with that will have a noticeable impact in bot / proxy / VPN traffic, especially if you don't have any data sets to test with the system. The query should look something like
http://check.getipintel.net/check.php?ip=IPHere&contact=SomeEmailAddressHere&flags=m
This option is the fastest.


flags=b
Use Dynamic Ban List and Dynamic Checks Only (Skip Some of the Bad IP Checks)
In this scenario, you want to use dynamic checks as well but you want to skip additional checks to see if the IP is a bad ip (see What do you mean by "Bad IP"?). In this mode, some bad IPs are still detected but the system does not attempt to go through the full bad IPs check because the time for the extra checks vary wildly (between an extra 200ms to 2 seconds). In this mode, false positives are more likely than dynamic ban lists only. Scores are lower compared to the full IP check (without any flag options) because less attributes are considered.

If you wish to use dynamic ban list and dynamic checks only, append the parameter &flags=b. This option is the best if dynamic ban lists isn't catching enough IPs but you don't want to run the full check because it takes too long and/or you want to have a predictable execution time. The query should look something like
http://check.getipintel.net/check.php?ip=IPHere&contact=SomeEmailAddressHere&flags=b
This option is slower than dynamic ban lists only, but much faster than the full check (no flags in query). This option is good if you only want proxy / VPN detection and you do not care about bad IPs, but flags=m is not catching enough proxy / VPN IPs.


Default Lookup (no flags)
Default Lookup
This is the default lookup with no flags. Since the system is designed to work with real-time systems (return a result as fast as possible), some time consuming checks are put into a background process. This allows the system to return a result much faster. If those time consuming checks reveal that the returned result was not accurate (which is rare), the system will adjust the values. However, you must query the service again with the same IP to obtain the new result. Typically, the background jobs take no longer than 5 seconds to complete. If you want to force the system to do a full lookup (no background processes), use flags=f option.


flags=f
Force Full Lookup
If you don't mind waiting up to 5 seconds for a result and you want the system to do a full lookup with one query, then use &flags=f option. The query should look something like
http://check.getipintel.net/check.php?ip=IPHere&contact=SomeEmailAddressHere&flags=f
This option is the slowest and should only be used on non-real-time applications.


oflags=b
Show if the IP is Considered as a Bad IP
Append &oflags=b to your query if you wish to know if the IP is considered as a bad IP. Standard output will append another integer value seperated by a comma. If JSON output is chosen, then an additional element called "BadIP" is added in the results. A value of 1 represents that it is a bad IP, 0 otheriwse. Note that if you are using flags option, the results may vary. For example, flags=b only does partial lookups for bad IP compared to flags=f. IPv6 is currently not supported yet for this feature. If the Proxy / VPN / Bad IP score for flags=m return 1 then oflags=b always returns 0 because if an IP is explicitly banned, then all bad IP checks are skipped.


oflags=c
Show which Country the IP belongs to
Append &oflags=c to your query if you wish to know which country the IP belongs to. Standard output will append the country seperated by a comma. If JSON output is chosen, then an additional element called "Country" is added in the results.
All countries are represented by a 2 character country code in ISO-3166 format.
Note that country information is directly pulled from RIRs. Accuracy will vary so consider this as an experimental feature.


oflags=i
Show if the IP is an iCloud Relay Egress IP, Google Cloud One VPN, or some other similiar service
Append &oflags=i to your query if you wish to know if the IP is an iCloud Relay Egress IP, Google One VPN IP, or some other similiar service. When more big technology companies implement these services, they will be added. JSON format is recommended as some information will not be available in standard format.
Standard output will append another integer value seperated by a comma.
If JSON output is chosen, then 3 additional elements are added.
  • iCloudRelayEgress A value of 1 represents that it is an iCloud Relay Egress IP, 0 otheriwse.
  • GoogleOneVPN A value of 1 represents that it is a Google One VPN IP, 0 otheriwse.
  • VPNType will display which type of VPN it is. Currently, there are only 4 possible outputs: none, GoogleOneVPN, GoogleFiVPN, and iCloudRelayEgress.
In the future, the two elements iCloudRelayEgress and GoogleOneVPN might be deprecated. Please use VPNType instead. When there's more big technology companies, it'll be included in the VPNType element.
Note that this feature is still experimental and if you find any issues, please contact me.


oflags=a
Show the ASN number of the IP
Append &oflags=a to your query if you wish to know the ASN number of the IP. If multiple ASNs are associated with the IP, it will be separated by a ; (semi-colon). Standard output will append the ASN value(s) seperated by a comma. If JSON output is chosen, then an additional element called "ASN" is added in the results.


format=json
Output Sample in JSON Format
Normally, the system outputs a negative value on error, or a value >=0 and <=1 on success. If you're more comfortable with JSON format, append the option &format=json to your queries.

An example of a query that's successful:
curl "http://check.getipintel.net/check.php?ip=66.228.119.72&contact=AValidEmailAddress&format=json&flags=m"
{"status":"success","result":"1","queryIP":"66.228.119.72","queryFlags":"m","queryFormat":"json","contact":"AValidEmailAddress"}


An example of a query that returns an error:
curl "http://check.getipintel.net/check.php?ip=10.10.10.10,8.8.8.8&contact=AValidEmailAddress&format=json"
{"status":"error","result":"-2","message":"Invalid IP address","queryIP":"10.10.10.10,8.8.8.8","queryFlags":null,"queryFormat":"json","contact":"AValidEmailAddress"}
Comparing the Different Flags

Flags Data Sets Used Pros Cons Response Time (No Network Latency) Suggested Use Based on Requirements
flags=m dynamic ban lists fastest, smallest chance for false positives IPs that are not on blocklists will get through <60 ms Least amount of false positives | fastest speeds | ok with letting some IPs through | only care about proxies & VPNs
flags=b dynamic ban lists, dynamic checks, some bad IP checks fast, catches more proxy / VPN IPs than flags=m, skips some compromised system detection so complaints from residential users are reduced because most likely the user do not know they're compromised or they received a dirty IP from their ISP higher chance of false positives than flags=m < 130 ms fast speeds, want to let less proxy / VPN IPs through than flags=m | do not want to fully utilize bad IP detection | only care about proxies & VPNs
no flags (default query) dynamic ban lists, dynamic checks, full bad IP checks fast, full IP check, a balance between speed and full IP check higher chance of false positives than flags=m | might require 1 more query after 5 seconds to be sure < 130 ms fast speeds, ok with making multiple queries with the same IP
flags=f dynamic ban lists, dynamic checks, full bad IP checks forces a full IP check which does not take additional queries to be sure higher chance of false positives than flags=m, slowest < 5000 ms ok with waiting for a full lookup that can take up to 5 secs

Error Codes
The proxy check system will return negative values on error. For standard format (non-json), an additional HTTP 400 status code is returned
  • -1 Invalid no input
  • -2 Invalid IP address
  • -3 Unroutable address / private address
  • -4 Unable to reach database, most likely the database is being updated. Keep an eye on twitter for more information.
  • -5 Your connecting IP has been banned from the system or you do not have permission to access a particular service. Did you exceed your query limits? Did you use an invalid email address? If you want more information, please use the contact links below.
  • -6 You did not provide any contact information with your query or the contact information is invalid.
  • If you exceed the number of allowed queries, you'll receive a HTTP 429 error.
Be sure to implement exception handling such as timeouts, HTTP 429 error, and the error codes listed above.

FAQs

Why is this service free?
I created this project because I couldn't find any good alternatives for a reasonable price. Since I have a masters degree in Computer Science specializing in Networking with interests in Machine Learning and NetSec, it's a fitting project for me to embark on. Compared to a popular paid service, the number of free queries that's being served by GetIPIntel translates to $60,000+/month and I've been told by a few people that GetIPIntel catches more proxies / VPN / bad IPs than said paid service. I'm offering it for free in the spirit of openess. Just because it's free, it does not mean it's bad, inaccurate, easy to develop, or easy to maintain. To keep things simple, please do not abuse this service as a free user and if you need more queries, contact me for a custom plan. If you're feeling generous and this API works well for you, please let your friends know.
Why is this different from similar services (even paid?)
There are many other services like this one that uses simple block lists, meaning a particular IP / IP block is specifically added or removed either manually or by code from various known/trusted sources. During a lookup, if the IP is on the list, then simply return the result accordingly. However, it's a very limited view because if the IP is not on a list, it doesn't mean it's not a proxy / VPN / bad IP. It means that the simple block list system does not know or have not come across that IP address. To claim an IP address is not a proxy / VPN / bad IP just because the system has never come across the IP is a logical fallacy (see Argument from Ignorance). GetIPIntel uses Machine Learning & Probability Theory techniques to infer on IPs it doesn't have explicit knowledge about (see What are dynamic checks?) and compute the output when you request it using up to date and large data sets. Thus, using a combination of block lists with dynamic checks will produce a more accurate result because the overall system is more intelligent.
What are dynamic checks?
Dynamic checks are used if the IP address is not explicitly listed in the static and dynamic files. The system attempts to retrieve characteristics (or attributes in Machine Learning terms) of the given IP. Based on that data, it uses concepts from Probability Theory and ML boosting techniques to generate an overall result. All results from dynamic checks are computed in real time using large & frequently updated datasets.
In short, dynamic checks allows the system to infer when it doesn't explicitly know if it's a proxy / vpn or not with mathematics.
What do you mean by "Bad IP"?
It refers any combination of crawlers / comment & email spammers / brute force attacks. IPs that are behaving "badly" in an automated manner. Networks that are infected with malware / trojans / botnet / etc are also considered "bad". It may be possible that the user is not aware that their systems are infected or they have received an IP by their ISP that was recently infected with malicious code. If you wish to skip this, see variations of implementation.
How many queries can I make?
There's a rate limit 15 requests / minute to prevent abuse as well as a burst parameter set to ensure smoothing of traffic. If you hit any of these limits, the web server will return a 429 error. Please do not exceed 500 queries per day. The limits may change based on abuse and/or server load which will be posted on twitter and at least one week in advance. If you need guaranteed resources and/or more queries, please contact me. In most cases, the cost is significantly less than other paid services.
What do you offer with your custom plans?
With custom plans, I can provide any amount of queries as a query pack that do not expire. All custom plans comes with a default of 300/queries per minute instead of 15 (could go higher if you want), automatic fail-over (2N redundancy), and dedicated resources. Please contact me via email (which is listed below) with your requirements.
Can I cache my results?
Of course but I do not recommend caching a particular value for more than 6 hours. The Internet drastically changes over a short period of time. Hijacked networks pop up and go away relatively quickly. A low scoring IP's behavior can change in a matter of seconds, as well as a high scoring IP. When the system detects an IP with high variance in previous scores, the probability will be recomputed live with the most up to date dataset for accuracy.
Sometimes the API will take longer than average to respond, why?
The free API is on a shared resource pool which means other people's actions can have an effect on your requests. All custom plans are on dedicated resources. If you're interested in one, please contact me.
Will corporate / business IPs generate a high score?
If the business ISP provides hosting or hosting related services on their network, then yes. See Variations of Implementation for a solution or just whitelist the IPs on your own system.
The IP looks like it's on a residential network but the score is high, why?
The IP is most likely a compromised system involved in spamming / brute-forcing / etc. It falls under the category of Bad IPs or it could perhaps be a public proxy that someone runs on their home computer. To find out if it's considered as a bad IP, use oflags=b.
What is an iCloud Egress proxy?
It is a service offered by Apple called iCloud Private relay. From a technical perspective, it is using an onion routing technique with at least one hop. According to Apple, "no single party - including apple - can view or collect the details of users’ browsing activity." However, if the activity of an Apple private relay user becomes extremely difficult to trace, it is even more important to be aware of these IPs because one cannot and should not assume that all iCloud Private relay users are using it in good faith. Furthermore, this has to be explicitly turned on by the user. To see if an IP is an Apple iCloud Private relay user / iCloud Egress proxy, use oflags=i.
What is a Google One VPN?
It is a VPN service offered by Google for Google One subscribers. According to Google, "VPN by Google One leverages advanced cryptographic techniques to ensure that no one, not even Google, can associate your network traffic with your account or identity." It is important to be aware of these IPs because one cannot assume that all Google One users are using the VPN in good faith. To see if an IP is a Google One VPN IP, use oflags=i.
What's whitelisted?
Known crawlers such as Google Bot, Yahoo bot, Bing bot, etc., as well as some known DNS resolvers are whitelisted and will return a result of 0. If you believe there's an error, please contact me.
Is SSL supported?
Yes, but be aware that the time to set up an SSL connection takes longer than a normal HTTP connection.
Why not look for HTTP headers, User agents, etc?
These values can easily be spoofed and therefore, unreliable.
I wrote some code but your server isn't responding. Why?
You might need to change the user-agent as cloudflare block certain connections with a weird user-agent. It shows up as "Bad Browser." If that doesn't work, please contact me and I'll look into the issue.
Code samples to get me started?
There's some code samples on my Github.
What information do you store?
Please see the privacy page for more information.



Disclaimer


No guarantees, warranties, etc, is provided or implied. Use at your own risk. GetIPIntel is not liable for damages or claims of any kind.




Terms of Service


By using this service, you agree to:
  • Not sell this service or information generated from this service, directly or indirectly, without explicit consent.
  • Not use / reuse information generated from this service, directly or indirectly, without giving credit to the source (this website).
  • Not exceed the query limits if you're a free user.
  • Not look up random IPs / incremental IP lookups. The database changes very often so information becomes stale very fast. It just causes a higher server load for no reason.
  • The Terms of Service may change at any given time, without prior notice.


Contact


You can find me on Twitter, GitHub, or email. If I do not respond to your email within 24 hours then something wrong, check your spam folder. Please send an email to my gmail address, or contact me via twitter. Ultimately, I want the system to be as accurate as possible, so please let me know if there are any inaccuracies, I'd like to fix the issue. Let me know if you have any custom requirements such as more queries per minute, skip cache so it always gets the latest data and recompute the result, etc.