initial commit

This commit is contained in:
Jörg Prante 2022-10-20 10:19:32 +02:00
commit 94d6e90a0d
442 changed files with 34467 additions and 0 deletions

16
.gitignore vendored Normal file
View file

@ -0,0 +1,16 @@
/.settings
/.classpath
/.project
/.gradle
**/data
**/work
**/logs
**/.idea
**/target
**/out
**/build
.DS_Store
*.iml
*~
*.key
*.crt

202
LICENSE.txt Normal file
View file

@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

68
README.md Normal file
View file

@ -0,0 +1,68 @@
# Java Net API for servers and clients
## A consolidated Uniform Resource Locator implementation for Java
A Uniform Resource Locator (URL) is a compact representation of the
location and access method for a resource available via the Internet.
Historically, there are many different forms of internet resource representations, for example,
the URL (RFC 1738 as of 1994), the URI (RFC 2396 as of 1998), and IRI (RFC 3987 as of 2005),
and most of them have updated specifications.
This Java implementation serves as a universal point of handling all
different forms. It follows the syntax of the Uniform Resource Identifier (RFC 3986)
in accordance with the https://url.spec.whatwg.org/[WHATWG URL standard].
This alternative implementation of Uniform Resource Locator combines the features of the vanilla URI/URL Java SDK implementations
but removes it peculiarities and deficiencies, such as `java.lang.IllegalArgumentException: Illegal character in path at ... at java.net.URI.create()`
Normalization, NIO charset encoding/decoding, IPv6, an extensive set of schemes, and path matching have been added.
Fast building and parsing URLs, improved percent decoding/encoding, and URI templating features are included, to make
this library also useful in URI and IRI contexts.
While parsing and building, you have better control about address resolving. Only explicit `resolveFromhost` methods
will execute host lookup queries against DNS resolvers, otherwise, no resolving will occur under the hood.
You can build URLs with a fluent API, for example
```
URL.http().host("foo.com").toUrlString()
```
And you can parse URLs with a fluent API, for exmaple
```
URL url = URL.parser().parse("file:///foo/bar?foo=bar#fragment");
```
There is no external dependency. The size of the jar library is ~118k. The only dependency on `java.net` are the classes
```
java.net.IDN
java.net.Inet4Address
java.net.Inet6Address
java.net.InetAddress
```
which might get re-implemented in another library at a later time, in a project like Netty DNS resolver.
## A simple HTTP server
## A netty-based HTTP server
# License
Copyright (C) 2018 Jörg Prante
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
you may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

46
build.gradle Normal file
View file

@ -0,0 +1,46 @@
buildscript {
repositories {
maven {
url 'https://xbib.org/repository'
}
}
dependencies {
classpath "org.xbib.gradle.plugin:gradle-plugin-shadow:1.1.1"
}
}
plugins {
id "de.marcphilipp.nexus-publish" version "0.4.0"
id "io.codearte.nexus-staging" version "0.21.1"
id "org.xbib.gradle.plugin.asciidoctor" version "2.5.2.1"
}
wrapper {
gradleVersion = libs.versions.gradle.get()
distributionType = Wrapper.DistributionType.ALL
}
ext {
user = 'xbib'
name = 'net'
description = 'Network classes for Java'
inceptionYear = '2016'
url = 'https://github.com/' + user + '/' + name
scmUrl = 'https://github.com/' + user + '/' + name
scmConnection = 'scm:git:git://github.com/' + user + '/' + name + '.git'
scmDeveloperConnection = 'scm:git:ssh://git@github.com:' + user + '/' + name + '.git'
issueManagementSystem = 'Github'
issueManagementUrl = ext.scmUrl + '/issues'
licenseName = 'The Apache License, Version 2.0'
licenseUrl = 'http://www.apache.org/licenses/LICENSE-2.0.txt'
}
subprojects {
apply from: rootProject.file('gradle/ide/idea.gradle')
apply from: rootProject.file('gradle/repositories/maven.gradle')
apply from: rootProject.file('gradle/compile/java.gradle')
apply from: rootProject.file('gradle/test/junit5.gradle')
apply from: rootProject.file('gradle/documentation/asciidoc.gradle')
apply from: rootProject.file('gradle/publish/maven.gradle')
}
apply from: rootProject.file('gradle/publish/sonatype.gradle')

View file

@ -0,0 +1,321 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE module PUBLIC
"-//Puppy Crawl//DTD Check Configuration 1.3//EN"
"http://www.puppycrawl.com/dtds/configuration_1_3.dtd">
<!-- This is a checkstyle configuration file. For descriptions of
what the following rules do, please see the checkstyle configuration
page at http://checkstyle.sourceforge.net/config.html -->
<module name="Checker">
<module name="FileTabCharacter">
<!-- Checks that there are no tab characters in the file.
-->
</module>
<module name="NewlineAtEndOfFile">
<property name="lineSeparator" value="lf"/>
</module>
<module name="RegexpSingleline">
<!-- Checks that FIXME is not used in comments. TODO is preferred.
-->
<property name="format" value="((//.*)|(\*.*))FIXME" />
<property name="message" value='TODO is preferred to FIXME. e.g. "TODO(johndoe): Refactor when v2 is released."' />
</module>
<module name="RegexpSingleline">
<!-- Checks that TODOs are named. (Actually, just that they are followed
by an open paren.)
-->
<property name="format" value="((//.*)|(\*.*))TODO[^(]" />
<property name="message" value='All TODOs should be named. e.g. "TODO(johndoe): Refactor when v2 is released."' />
</module>
<module name="JavadocPackage">
<!-- Checks that each Java package has a Javadoc file used for commenting.
Only allows a package-info.java, not package.html. -->
</module>
<!-- All Java AST specific tests live under TreeWalker module. -->
<module name="TreeWalker">
<!--
IMPORT CHECKS
-->
<module name="RedundantImport">
<!-- Checks for redundant import statements. -->
<property name="severity" value="error"/>
</module>
<module name="ImportOrder">
<property name="separated" value="true"/>
<property name="severity" value="warning"/>
<property name="groups" value="*,javax,java"/>
<property name="option" value="bottom"/>
<property name="elements" value="IMPORT, STATIC_IMPORT"/>
</module>
<!--
JAVADOC CHECKS
-->
<!-- Checks for Javadoc comments. -->
<!-- See http://checkstyle.sf.net/config_javadoc.html -->
<module name="JavadocMethod">
<property name="scope" value="protected"/>
<property name="severity" value="warning"/>
<property name="allowMissingJavadoc" value="true"/>
<property name="allowMissingParamTags" value="true"/>
<property name="allowMissingReturnTag" value="true"/>
<property name="allowMissingThrowsTags" value="true"/>
<property name="allowThrowsTagsForSubclasses" value="true"/>
<property name="allowUndeclaredRTE" value="true"/>
</module>
<module name="JavadocType">
<property name="scope" value="protected"/>
<property name="severity" value="error"/>
</module>
<module name="JavadocStyle">
<property name="severity" value="warning"/>
</module>
<!--
NAMING CHECKS
-->
<!-- Item 38 - Adhere to generally accepted naming conventions -->
<module name="PackageName">
<!-- Validates identifiers for package names against the
supplied expression. -->
<!-- Here the default checkstyle rule restricts package name parts to
seven characters, this is not in line with common practice at Google.
-->
<property name="format" value="^[a-z]+(\.[a-z][a-z0-9]{1,})*$"/>
<property name="severity" value="warning"/>
</module>
<module name="TypeNameCheck">
<!-- Validates static, final fields against the
expression "^[A-Z][a-zA-Z0-9]*$". -->
<metadata name="altname" value="TypeName"/>
<property name="severity" value="warning"/>
</module>
<module name="ConstantNameCheck">
<!-- Validates non-private, static, final fields against the supplied
public/package final fields "^[A-Z][A-Z0-9]*(_[A-Z0-9]+)*$". -->
<metadata name="altname" value="ConstantName"/>
<property name="applyToPublic" value="true"/>
<property name="applyToProtected" value="true"/>
<property name="applyToPackage" value="true"/>
<property name="applyToPrivate" value="false"/>
<property name="format" value="^([A-Z][A-Z0-9]*(_[A-Z0-9]+)*|FLAG_.*)$"/>
<message key="name.invalidPattern"
value="Variable ''{0}'' should be in ALL_CAPS (if it is a constant) or be private (otherwise)."/>
<property name="severity" value="warning"/>
</module>
<module name="StaticVariableNameCheck">
<!-- Validates static, non-final fields against the supplied
expression "^[a-z][a-zA-Z0-9]*_?$". -->
<metadata name="altname" value="StaticVariableName"/>
<property name="applyToPublic" value="true"/>
<property name="applyToProtected" value="true"/>
<property name="applyToPackage" value="true"/>
<property name="applyToPrivate" value="true"/>
<property name="format" value="^[a-z][a-zA-Z0-9]*_?$"/>
<property name="severity" value="warning"/>
</module>
<module name="MemberNameCheck">
<!-- Validates non-static members against the supplied expression. -->
<metadata name="altname" value="MemberName"/>
<property name="applyToPublic" value="true"/>
<property name="applyToProtected" value="true"/>
<property name="applyToPackage" value="true"/>
<property name="applyToPrivate" value="true"/>
<property name="format" value="^[a-z][a-zA-Z0-9]*$"/>
<property name="severity" value="warning"/>
</module>
<module name="MethodNameCheck">
<!-- Validates identifiers for method names. -->
<metadata name="altname" value="MethodName"/>
<property name="format" value="^[a-z][a-zA-Z0-9]*(_[a-zA-Z0-9]+)*$"/>
<property name="severity" value="warning"/>
</module>
<module name="ParameterName">
<!-- Validates identifiers for method parameters against the
expression "^[a-z][a-zA-Z0-9]*$". -->
<property name="severity" value="warning"/>
</module>
<module name="LocalFinalVariableName">
<!-- Validates identifiers for local final variables against the
expression "^[a-z][a-zA-Z0-9]*$". -->
<property name="severity" value="warning"/>
</module>
<module name="LocalVariableName">
<!-- Validates identifiers for local variables against the
expression "^[a-z][a-zA-Z0-9]*$". -->
<property name="severity" value="warning"/>
</module>
<!--
LENGTH and CODING CHECKS
-->
<module name="LineLength">
<!-- Checks if a line is too long. -->
<property name="max" value="${com.puppycrawl.tools.checkstyle.checks.sizes.LineLength.max}" default="128"/>
<property name="severity" value="error"/>
<!--
The default ignore pattern exempts the following elements:
- import statements
- long URLs inside comments
-->
<property name="ignorePattern"
value="${com.puppycrawl.tools.checkstyle.checks.sizes.LineLength.ignorePattern}"
default="^(package .*;\s*)|(import .*;\s*)|( *(\*|//).*https?://.*)$"/>
</module>
<module name="LeftCurly">
<!-- Checks for placement of the left curly brace ('{'). -->
<property name="severity" value="warning"/>
</module>
<module name="RightCurly">
<!-- Checks right curlies on CATCH, ELSE, and TRY blocks are on
the same line. e.g., the following example is fine:
<pre>
if {
...
} else
</pre>
-->
<!-- This next example is not fine:
<pre>
if {
...
}
else
</pre>
-->
<property name="option" value="same"/>
<property name="severity" value="warning"/>
</module>
<!-- Checks for braces around if and else blocks -->
<module name="NeedBraces">
<property name="severity" value="warning"/>
<property name="elements" value="LITERAL_IF, LITERAL_ELSE, LITERAL_FOR, LITERAL_WHILE, LITERAL_DO"/>
</module>
<module name="UpperEll">
<!-- Checks that long constants are defined with an upper ell.-->
<property name="severity" value="error"/>
</module>
<module name="FallThrough">
<!-- Warn about falling through to the next case statement. Similar to
javac -Xlint:fallthrough, but the check is suppressed if a single-line comment
on the last non-blank line preceding the fallen-into case contains 'fall through' (or
some other variants which we don't publicized to promote consistency).
-->
<property name="reliefPattern"
value="fall through|Fall through|fallthru|Fallthru|falls through|Falls through|fallthrough|Fallthrough|No break|NO break|no break|continue on"/>
<property name="severity" value="error"/>
</module>
<!--
MODIFIERS CHECKS
-->
<module name="ModifierOrder">
<!-- Warn if modifier order is inconsistent with JLS3 8.1.1, 8.3.1, and
8.4.3. The prescribed order is:
public, protected, private, abstract, static, final, transient, volatile,
synchronized, native, strictfp
-->
</module>
<!--
WHITESPACE CHECKS
-->
<module name="WhitespaceAround">
<!-- Checks that various elements are surrounded by whitespace.
This includes most binary operators and keywords followed
by regular or curly braces.
-->
<property name="elements" value="ASSIGN, BAND, BAND_ASSIGN, BOR,
BOR_ASSIGN, BSR, BSR_ASSIGN, BXOR, BXOR_ASSIGN, COLON, DIV, DIV_ASSIGN,
EQUAL, GE, GT, LAND, LE, LITERAL_CATCH, LITERAL_DO, LITERAL_ELSE,
LITERAL_FINALLY, LITERAL_FOR, LITERAL_IF, LITERAL_RETURN,
LITERAL_SYNCHRONIZED, LITERAL_TRY, LITERAL_WHILE, LOR, LT, MINUS,
MINUS_ASSIGN, MOD, MOD_ASSIGN, NOT_EQUAL, PLUS, PLUS_ASSIGN, QUESTION,
SL, SL_ASSIGN, SR_ASSIGN, STAR, STAR_ASSIGN"/>
<property name="severity" value="error"/>
</module>
<module name="WhitespaceAfter">
<!-- Checks that commas, semicolons and typecasts are followed by
whitespace.
-->
<property name="elements" value="COMMA, SEMI, TYPECAST"/>
</module>
<module name="NoWhitespaceAfter">
<!-- Checks that there is no whitespace after various unary operators.
Linebreaks are allowed.
-->
<property name="elements" value="BNOT, DEC, DOT, INC, LNOT, UNARY_MINUS,
UNARY_PLUS"/>
<property name="allowLineBreaks" value="true"/>
<property name="severity" value="error"/>
</module>
<module name="NoWhitespaceBefore">
<!-- Checks that there is no whitespace before various unary operators.
Linebreaks are allowed.
-->
<property name="elements" value="SEMI, DOT, POST_DEC, POST_INC"/>
<property name="allowLineBreaks" value="true"/>
<property name="severity" value="error"/>
</module>
<module name="ParenPad">
<!-- Checks that there is no whitespace before close parens or after
open parens.
-->
<property name="severity" value="warning"/>
</module>
</module>
</module>

5
gradle.properties Normal file
View file

@ -0,0 +1,5 @@
group = org.xbib
name = net
version = 3.0.0
org.gradle.warning.mode = ALL

View file

@ -0,0 +1,30 @@
apply plugin: 'java-library'
java {
modularity.inferModulePath.set(true)
}
compileJava {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
compileTestJava {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
jar {
manifest {
attributes('Implementation-Version': project.version)
}
duplicatesStrategy = DuplicatesStrategy.INCLUDE
}
tasks.withType(JavaCompile) {
options.compilerArgs.add('-Xlint:all,-exports')
}
javadoc {
options.addStringOption('Xdoclint:none', '-quiet')
}

View file

@ -0,0 +1,13 @@
apply plugin: 'org.xbib.gradle.plugin.asciidoctor'
asciidoctor {
attributes 'source-highlighter': 'coderay',
toc: 'left',
doctype: 'book',
icons: 'font',
encoding: 'utf-8',
sectlink: true,
sectanchors: true,
linkattrs: true,
imagesdir: 'img'
}

8
gradle/ide/idea.gradle Normal file
View file

@ -0,0 +1,8 @@
apply plugin: 'idea'
idea {
module {
outputDir file('build/classes/java/main')
testOutputDir file('build/classes/java/test')
}
}

27
gradle/publish/ivy.gradle Normal file
View file

@ -0,0 +1,27 @@
apply plugin: 'ivy-publish'
publishing {
repositories {
ivy {
url = "https://xbib.org/repo"
}
}
publications {
ivy(IvyPublication) {
from components.java
descriptor {
license {
name = 'The Apache License, Version 2.0'
url = 'http://www.apache.org/licenses/LICENSE-2.0.txt'
}
author {
name = 'Jörg Prante'
url = 'http://example.com/users/jane'
}
descriptor.description {
text = rootProject.ext.description
}
}
}
}
}

View file

@ -0,0 +1,64 @@
apply plugin: "de.marcphilipp.nexus-publish"
publishing {
publications {
mavenJava(MavenPublication) {
from components.java
pom {
name = project.name
description = rootProject.ext.description
url = rootProject.ext.url
inceptionYear = rootProject.ext.inceptionYear
packaging = 'jar'
organization {
name = 'xbib'
url = 'https://xbib.org'
}
developers {
developer {
id = 'jprante'
name = 'Jörg Prante'
email = 'joergprante@gmail.com'
url = 'https://github.com/jprante'
}
}
scm {
url = rootProject.ext.scmUrl
connection = rootProject.ext.scmConnection
developerConnection = rootProject.ext.scmDeveloperConnection
}
issueManagement {
system = rootProject.ext.issueManagementSystem
url = rootProject.ext.issueManagementUrl
}
licenses {
license {
name = rootProject.ext.licenseName
url = rootProject.ext.licenseUrl
distribution = 'repo'
}
}
}
}
}
}
if (project.hasProperty("signing.keyId")) {
apply plugin: 'signing'
signing {
sign publishing.publications.mavenJava
}
}
if (project.hasProperty('ossrhUsername') && project.hasProperty('ossrhPassword')) {
nexusPublishing {
repositories {
sonatype {
username = project.property('ossrhUsername')
password = project.property('ossrhPassword')
packageGroup = "org.xbib"
}
}
}
}

View file

@ -0,0 +1,11 @@
if (project.hasProperty('ossrhUsername') && project.hasProperty('ossrhPassword')) {
apply plugin: 'io.codearte.nexus-staging'
nexusStaging {
username = project.property('ossrhUsername')
password = project.property('ossrhPassword')
packageGroup = "org.xbib"
}
}

View file

@ -0,0 +1,50 @@
subprojects {
sonarqube {
properties {
property "sonar.projectName", "${project.group} ${project.name}"
property "sonar.sourceEncoding", "UTF-8"
property "sonar.tests", "src/test/java"
property "sonar.scm.provider", "git"
property "sonar.junit.reportsPath", "build/test-results/test/"
}
}
tasks.withType(Checkstyle) {
ignoreFailures = true
reports {
xml.enabled = true
html.enabled = true
}
}
tasks.withType(Pmd) {
ignoreFailures = true
reports {
xml.enabled = true
html.enabled = true
}
}
checkstyle {
//configFile = rootProject.file('config/checkstyle/checkstyle.xml')
ignoreFailures = true
showViolations = true
}
spotbugs {
effort = "max"
reportLevel = "low"
//includeFilter = file("findbugs-exclude.xml")
}
tasks.withType(com.github.spotbugs.SpotBugsTask) {
ignoreFailures = true
reports {
xml.enabled = false
html.enabled = true
}
}
}

View file

@ -0,0 +1,4 @@
repositories {
mavenLocal()
mavenCentral()
}

22
gradle/test/jmh.gradle Normal file
View file

@ -0,0 +1,22 @@
sourceSets {
jmh {
java.srcDirs = ['src/jmh/java']
resources.srcDirs = ['src/jmh/resources']
compileClasspath += sourceSets.main.runtimeClasspath
}
}
dependencies {
jmhImplementation 'org.openjdk.jmh:jmh-core:1.34'
jmhAnnotationProcessor 'org.openjdk.jmh:jmh-generator-annprocess:1.34'
}
task jmh(type: JavaExec, group: 'jmh', dependsOn: jmhClasses) {
mainClass.set('org.openjdk.jmh.Main')
classpath = sourceSets.jmh.compileClasspath + sourceSets.jmh.runtimeClasspath
project.file('build/reports/jmh').mkdirs()
args '-rf', 'json'
args '-rff', project.file('build/reports/jmh/result.json')
}
classes.finalizedBy(jmhClasses)

36
gradle/test/junit5.gradle Normal file
View file

@ -0,0 +1,36 @@
dependencies {
testImplementation libs.junit.jupiter.api
testImplementation libs.junit.jupiter.params
testImplementation libs.hamcrest
testRuntimeOnly libs.junit.jupiter.engine
}
test {
useJUnitPlatform()
failFast = false
jvmArgs '--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED',
'--add-exports=java.base/jdk.internal.ref=ALL-UNNAMED',
'--add-exports=java.base/sun.nio.ch=ALL-UNNAMED',
'--add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED',
'--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED',
'--add-opens=jdk.compiler/com.sun.tools.javac=ALL-UNNAMED',
'--add-opens=java.base/java.lang=ALL-UNNAMED',
'--add-opens=java.base/java.lang.reflect=ALL-UNNAMED',
'--add-opens=java.base/java.io=ALL-UNNAMED',
'--add-opens=java.base/java.nio=ALL-UNNAMED',
'--add-opens=java.base/java.util=ALL-UNNAMED'
systemProperty 'java.util.logging.config.file', 'src/test/resources/logging.properties'
systemProperty 'io.netty.tryReflectionSetAccessible', 'true'
testLogging {
events 'STARTED', 'PASSED', 'FAILED', 'SKIPPED'
}
afterSuite { desc, result ->
if (!desc.parent) {
println "\nTest result: ${result.resultType}"
println "Test summary: ${result.testCount} tests, " +
"${result.successfulTestCount} succeeded, " +
"${result.failedTestCount} failed, " +
"${result.skippedTestCount} skipped"
}
}
}

BIN
gradle/wrapper/gradle-wrapper.jar vendored Normal file

Binary file not shown.

View file

@ -0,0 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.5.1-all.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

240
gradlew vendored Executable file
View file

@ -0,0 +1,240 @@
#!/bin/sh
#
# Copyright © 2015-2021 the original authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
##############################################################################
#
# Gradle start up script for POSIX generated by Gradle.
#
# Important for running:
#
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
# noncompliant, but you have some other compliant shell such as ksh or
# bash, then to run this script, type that shell name before the whole
# command line, like:
#
# ksh Gradle
#
# Busybox and similar reduced shells will NOT work, because this script
# requires all of these POSIX shell features:
# * functions;
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
# * compound commands having a testable exit status, especially «case»;
# * various built-in commands including «command», «set», and «ulimit».
#
# Important for patching:
#
# (2) This script targets any POSIX shell, so it avoids extensions provided
# by Bash, Ksh, etc; in particular arrays are avoided.
#
# The "traditional" practice of packing multiple parameters into a
# space-separated string is a well documented source of bugs and security
# problems, so this is (mostly) avoided, by progressively accumulating
# options in "$@", and eventually passing that to Java.
#
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
# see the in-line comments for details.
#
# There are tweaks for specific operating systems such as AIX, CygWin,
# Darwin, MinGW, and NonStop.
#
# (3) This script is generated from the Groovy template
# https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
# within the Gradle project.
#
# You can find Gradle at https://github.com/gradle/gradle/.
#
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
app_path=$0
# Need this for daisy-chained symlinks.
while
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
[ -h "$app_path" ]
do
ls=$( ls -ld "$app_path" )
link=${ls#*' -> '}
case $link in #(
/*) app_path=$link ;; #(
*) app_path=$APP_HOME$link ;;
esac
done
APP_HOME=$( cd "${APP_HOME:-./}" && pwd -P ) || exit
APP_NAME="Gradle"
APP_BASE_NAME=${0##*/}
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD=maximum
warn () {
echo "$*"
} >&2
die () {
echo
echo "$*"
echo
exit 1
} >&2
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "$( uname )" in #(
CYGWIN* ) cygwin=true ;; #(
Darwin* ) darwin=true ;; #(
MSYS* | MINGW* ) msys=true ;; #(
NONSTOP* ) nonstop=true ;;
esac
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD=$JAVA_HOME/jre/sh/java
else
JAVACMD=$JAVA_HOME/bin/java
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD=java
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
# Increase the maximum file descriptors if we can.
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
case $MAX_FD in #(
max*)
MAX_FD=$( ulimit -H -n ) ||
warn "Could not query maximum file descriptor limit"
esac
case $MAX_FD in #(
'' | soft) :;; #(
*)
ulimit -n "$MAX_FD" ||
warn "Could not set maximum file descriptor limit to $MAX_FD"
esac
fi
# Collect all arguments for the java command, stacking in reverse order:
# * args from the command line
# * the main class name
# * -classpath
# * -D...appname settings
# * --module-path (only if needed)
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
# For Cygwin or MSYS, switch paths to Windows format before running java
if "$cygwin" || "$msys" ; then
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
JAVACMD=$( cygpath --unix "$JAVACMD" )
# Now convert the arguments - kludge to limit ourselves to /bin/sh
for arg do
if
case $arg in #(
-*) false ;; # don't mess with options #(
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
[ -e "$t" ] ;; #(
*) false ;;
esac
then
arg=$( cygpath --path --ignore --mixed "$arg" )
fi
# Roll the args list around exactly as many times as the number of
# args, so each arg winds up back in the position where it started, but
# possibly modified.
#
# NB: a `for` loop captures its iteration list before it begins, so
# changing the positional parameters here affects neither the number of
# iterations, nor the values presented in `arg`.
shift # remove old arg
set -- "$@" "$arg" # push replacement arg
done
fi
# Collect all arguments for the java command;
# * $DEFAULT_JVM_OPTS, $JAVA_OPTS, and $GRADLE_OPTS can contain fragments of
# shell script including quotes and variable substitutions, so put them in
# double quotes to make sure that they get re-expanded; and
# * put everything else in single quotes, so that it's not re-expanded.
set -- \
"-Dorg.gradle.appname=$APP_BASE_NAME" \
-classpath "$CLASSPATH" \
org.gradle.wrapper.GradleWrapperMain \
"$@"
# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi
# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
#
# In Bash we could simply go:
#
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
# set -- "${ARGS[@]}" "$@"
#
# but POSIX shell has neither arrays nor command substitution, so instead we
# post-process each arg (as a line of input to sed) to backslash-escape any
# character that might be a shell metacharacter, then use eval to reverse
# that process (while maintaining the separation between arguments), and wrap
# the whole thing up as a single "set" statement.
#
# This will of course break if any of these variables contains a newline or
# an unmatched quote.
#
eval "set -- $(
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
xargs -n1 |
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
tr '\n' ' '
)" '"$@"'
exec "$JAVACMD" "$@"

91
gradlew.bat vendored Normal file
View file

@ -0,0 +1,91 @@
@rem
@rem Copyright 2015 the original author or authors.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.
@rem
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables with windows NT shell
if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0
if "%DIRNAME%"=="" set DIRNAME=.
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if %ERRORLEVEL% equ 0 goto execute
echo.
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
echo.
echo Please set the JAVA_HOME variable in your environment to match the
echo location of your Java installation.
goto fail
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto execute
echo.
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME%
echo.
echo Please set the JAVA_HOME variable in your environment to match the
echo location of your Java installation.
goto fail
:execute
@rem Setup the command line
set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
@rem Execute Gradle
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %*
:end
@rem End local scope for the variables with windows NT shell
if %ERRORLEVEL% equ 0 goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%
:mainEnd
if "%OS%"=="Windows_NT" endlocal
:omega

View file

@ -0,0 +1,7 @@
dependencies {
api project(':net-http-client-netty')
api libs.net.security
api libs.netty.codec.http2
api libs.netty.handler.proxy
testImplementation project(':net-http-netty-boringssl')
}

View file

@ -0,0 +1,27 @@
import org.xbib.net.http.client.netty.HttpChannelInitializer;
import org.xbib.net.http.client.netty.secure.ClientSecureSocketProvider;
import org.xbib.net.http.client.netty.secure.JdkClientSecureSocketProvider;
import org.xbib.net.http.client.netty.secure.http1.Https1ChannelInitializer;
import org.xbib.net.http.client.netty.secure.http2.Https2ChannelInitializer;
module org.xbib.net.http.client.netty.secure {
exports org.xbib.net.http.client.netty.secure;
exports org.xbib.net.http.client.netty.secure.http1;
exports org.xbib.net.http.client.netty.secure.http2;
requires org.xbib.net;
requires org.xbib.net.http;
requires org.xbib.net.http.client;
requires org.xbib.net.http.client.netty;
requires org.xbib.net.security;
requires io.netty.handler;
requires io.netty.codec.http;
requires io.netty.codec.http2;
requires io.netty.handler.proxy;
requires io.netty.transport;
requires java.logging;
requires io.netty.common;
uses ClientSecureSocketProvider;
provides ClientSecureSocketProvider with JdkClientSecureSocketProvider;
uses HttpChannelInitializer;
provides HttpChannelInitializer with Https1ChannelInitializer, Https2ChannelInitializer;
}

View file

@ -0,0 +1,21 @@
package org.xbib.net.http.client.netty.secure;
import io.netty.handler.ssl.CipherSuiteFilter;
import io.netty.handler.ssl.SslProvider;
import java.security.Provider;
import org.xbib.net.http.HttpAddress;
public interface ClientSecureSocketProvider {
String name();
Provider securityProvider(HttpAddress address);
SslProvider sslProvider(HttpAddress address);
Iterable<String> ciphers(HttpAddress address);
CipherSuiteFilter cipherSuiteFilter(HttpAddress address);
String[] protocols(HttpAddress address);
}

View file

@ -0,0 +1,23 @@
package org.xbib.net.http.client.netty.secure;
import javax.net.ssl.SSLSession;
import org.xbib.net.http.HttpHeaders;
import org.xbib.net.http.client.netty.HttpRequest;
public class HttpsRequest extends HttpRequest {
private final HttpsRequestBuilder builder;
protected HttpsRequest(HttpsRequestBuilder builder, HttpHeaders headers) {
super(builder, headers);
this.builder = builder;
}
public static HttpsRequestBuilder builder() {
return new HttpsRequestBuilder();
}
public SSLSession getSSLSession() {
return builder.sslSession;
}
}

View file

@ -0,0 +1,18 @@
package org.xbib.net.http.client.netty.secure;
import javax.net.ssl.SSLSession;
import org.xbib.net.http.client.netty.HttpRequestBuilder;
public class HttpsRequestBuilder extends HttpRequestBuilder {
SSLSession sslSession;
public HttpsRequestBuilder setSSLSession(SSLSession sslSession) {
this.sslSession = sslSession;
return this;
}
public HttpsRequest build() {
return new HttpsRequest(this, validateHeaders());
}
}

View file

@ -0,0 +1,22 @@
package org.xbib.net.http.client.netty.secure;
import javax.net.ssl.SSLSession;
import org.xbib.net.http.client.netty.HttpResponse;
public class HttpsResponse extends HttpResponse {
private final HttpsResponseBuilder builder;
protected HttpsResponse(HttpsResponseBuilder builder) {
super(builder);
this.builder = builder;
}
public static HttpsResponseBuilder builder() {
return new HttpsResponseBuilder();
}
public SSLSession getSSLSession() {
return builder.sslSession;
}
}

View file

@ -0,0 +1,20 @@
package org.xbib.net.http.client.netty.secure;
import javax.net.ssl.SSLSession;
import org.xbib.net.http.client.netty.HttpResponseBuilder;
public class HttpsResponseBuilder extends HttpResponseBuilder {
SSLSession sslSession;
public HttpsResponseBuilder setSSLSession(SSLSession sslSession) {
this.sslSession = sslSession;
return this;
}
@Override
public HttpsResponse build() {
super.build();
return new HttpsResponse(this);
}
}

View file

@ -0,0 +1,50 @@
package org.xbib.net.http.client.netty.secure;
import io.netty.handler.ssl.CipherSuiteFilter;
import io.netty.handler.ssl.SslProvider;
import io.netty.handler.ssl.SupportedCipherSuiteFilter;
import java.security.Provider;
import java.util.Arrays;
import javax.net.ssl.SSLSocketFactory;
import org.xbib.net.http.HttpAddress;
public class JdkClientSecureSocketProvider implements ClientSecureSocketProvider {
// https://convincingbits.wordpress.com/2016/02/17/ssl-tls-with-java-7-and-the-death-of-sslv2hello/
static {
System.setProperty("https.protocol", "TLSv1");
}
public JdkClientSecureSocketProvider() {
}
@Override
public String name() {
return "JDK";
}
@Override
public Provider securityProvider(HttpAddress httpAddress) {
return null;
}
@Override
public SslProvider sslProvider(HttpAddress httpAddress) {
return SslProvider.JDK;
}
@Override
public Iterable<String> ciphers(HttpAddress httpAddress) {
return Arrays.asList(((SSLSocketFactory) SSLSocketFactory.getDefault()).getDefaultCipherSuites());
}
@Override
public CipherSuiteFilter cipherSuiteFilter(HttpAddress httpAddress) {
return SupportedCipherSuiteFilter.INSTANCE;
}
@Override
public String[] protocols(HttpAddress httpAddress) {
return new String[] { "TLSv1.3", "TLSv1.2" };
}
}

View file

@ -0,0 +1,159 @@
package org.xbib.net.http.client.netty.secure;
import io.netty.handler.ssl.SslHandler;
import io.netty.handler.ssl.util.InsecureTrustManagerFactory;
import io.netty.util.AttributeKey;
import java.io.InputStream;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.net.ssl.TrustManagerFactory;
import org.xbib.net.http.client.ClientAuthMode;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
public class NettyHttpsClientConfig extends NettyHttpClientConfig {
private static final Logger logger = Logger.getLogger(NettyHttpsClientConfig.class.getName());
public static final AttributeKey<SslHandler> ATTRIBUTE_KEY_SSL_HANDLER = AttributeKey.valueOf("_ssl_handler");
private static TrustManagerFactory TRUST_MANAGER_FACTORY;
static {
try {
TRUST_MANAGER_FACTORY = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
} catch (Exception e) {
TRUST_MANAGER_FACTORY = null;
}
}
private TrustManagerFactory trustManagerFactory = TRUST_MANAGER_FACTORY;
private String secureSocketProviderName = "JDK";
private KeyStore trustManagerKeyStore = null;
private ClientAuthMode clientAuthMode = ClientAuthMode.NONE;
private InputStream keyCertChainInputStream;
private InputStream keyInputStream;
private String keyPassword;
private boolean protocolNegotiationEnabled = false;
/*
* Automatically selects the protocol from our secure socket providers.
*/
private String[] secureProtocolName = null;
public NettyHttpsClientConfig() {
}
public NettyHttpsClientConfig setTrustManagerFactory(TrustManagerFactory trustManagerFactory) {
this.trustManagerFactory = trustManagerFactory;
return this;
}
public NettyHttpsClientConfig trustInsecure() {
this.trustManagerFactory = InsecureTrustManagerFactory.INSTANCE;
return this;
}
public TrustManagerFactory getTrustManagerFactory() {
initializeTrustManagerFactory();
return trustManagerFactory;
}
public NettyHttpsClientConfig setTrustManagerKeyStore(KeyStore trustManagerKeyStore) {
this.trustManagerKeyStore = trustManagerKeyStore;
return this;
}
public KeyStore getTrustManagerKeyStore() {
return trustManagerKeyStore;
}
public NettyHttpsClientConfig setSecureSocketProviderName(String secureSocketProviderName) {
this.secureSocketProviderName = secureSocketProviderName;
return this;
}
public String getSecureSocketProviderName() {
return secureSocketProviderName;
}
public NettyHttpsClientConfig setSecureProtocolName(String[] secureProtocolName) {
this.secureProtocolName = secureProtocolName;
return this;
}
public String[] getSecureProtocolName() {
return secureProtocolName;
}
public NettyHttpsClientConfig setKeyCert(InputStream keyCertChainInputStream,
InputStream keyInputStream) {
this.keyCertChainInputStream = keyCertChainInputStream;
this.keyInputStream = keyInputStream;
return this;
}
public InputStream getKeyCertChainInputStream() {
return keyCertChainInputStream;
}
public InputStream getKeyInputStream() {
return keyInputStream;
}
public NettyHttpsClientConfig setKeyCert(InputStream keyCertChainInputStream,
InputStream keyInputStream,
String keyPassword) {
this.keyCertChainInputStream = keyCertChainInputStream;
this.keyInputStream = keyInputStream;
this.keyPassword = keyPassword;
return this;
}
public String getKeyPassword() {
return keyPassword;
}
public NettyHttpsClientConfig setClientAuthMode(ClientAuthMode clientAuthMode) {
this.clientAuthMode = clientAuthMode;
return this;
}
public ClientAuthMode getClientAuthMode() {
return clientAuthMode;
}
public NettyHttpsClientConfig setProtocolNegotiation(boolean negotiationEnabled) {
this.protocolNegotiationEnabled = negotiationEnabled;
return this;
}
public boolean isProtocolNegotiationEnabled() {
return protocolNegotiationEnabled;
}
/**
* Initialize trust manager factory once per client lifecycle.
*/
private void initializeTrustManagerFactory() {
if (trustManagerFactory != null) {
try {
trustManagerFactory.init(trustManagerKeyStore);
logger.log(Level.FINE, "trust manager factory initialized with key store " + trustManagerFactory);
} catch (KeyStoreException e) {
logger.log(Level.WARNING, e.getMessage(), e);
}
} else {
logger.log(Level.INFO, "no trust manager factory present");
}
}
}

View file

@ -0,0 +1,260 @@
package org.xbib.net.http.client.netty.secure.http1;
import io.netty.channel.Channel;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.handler.codec.http.HttpClientCodec;
import io.netty.handler.codec.http.HttpContentDecompressor;
import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.codec.http2.Http2FrameLogger;
import io.netty.handler.codec.http2.Http2MultiplexCodec;
import io.netty.handler.codec.http2.Http2MultiplexCodecBuilder;
import io.netty.handler.codec.http2.Http2Settings;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.proxy.Socks5ProxyHandler;
import io.netty.handler.ssl.ApplicationProtocolConfig;
import io.netty.handler.ssl.ApplicationProtocolNames;
import io.netty.handler.ssl.ApplicationProtocolNegotiationHandler;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.SslHandler;
import io.netty.handler.stream.ChunkedWriteHandler;
import io.netty.handler.timeout.IdleStateHandler;
import io.netty.handler.timeout.ReadTimeoutHandler;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.net.InetSocketAddress;
import java.security.Provider;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.ServiceLoader;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.net.ssl.SNIHostName;
import javax.net.ssl.SNIServerName;
import javax.net.ssl.SSLEngine;
import javax.net.ssl.SSLParameters;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpVersion;
import org.xbib.net.http.client.netty.HttpChannelInitializer;
import org.xbib.net.http.client.netty.Interaction;
import org.xbib.net.http.client.netty.NettyCustomizer;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.http2.Http2Messages;
import org.xbib.net.http.client.netty.http1.Http1Handler;
import org.xbib.net.http.client.netty.TrafficLoggingHandler;
import org.xbib.net.http.client.netty.secure.ClientSecureSocketProvider;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
public class Https1ChannelInitializer implements HttpChannelInitializer {
private static final Logger logger = Logger.getLogger(Https1ChannelInitializer.class.getName());
public Https1ChannelInitializer() {
}
@Override
public boolean supports(HttpAddress address) {
return HttpVersion.HTTP_1_1.equals(address.getVersion()) && address.isSecure();
}
@Override
public Interaction newInteraction(NettyHttpClient client, HttpAddress httpAddress) {
return new Https1Interaction(client, httpAddress);
}
@Override
public void init(Channel channel,
HttpAddress httpAddress,
NettyHttpClient nettyHttpClient,
NettyCustomizer nettyCustomizer,
Interaction interaction) throws IOException {
NettyHttpsClientConfig nettyHttpClientConfig = (NettyHttpsClientConfig) nettyHttpClient.getClientConfig();
ChannelPipeline pipeline = channel.pipeline();
if (nettyHttpClientConfig.isDebug()) {
pipeline.addLast("client-traffic", new TrafficLoggingHandler(LogLevel.DEBUG));
}
int readTimeoutMilllis = nettyHttpClientConfig.getSocketConfig().getReadTimeoutMillis();
if (readTimeoutMilllis > 0) {
pipeline.addLast("client-read-timeout", new ReadTimeoutHandler(readTimeoutMilllis / 1000));
}
int socketTimeoutMillis = nettyHttpClientConfig.getSocketConfig().getSocketTimeoutMillis();
if (socketTimeoutMillis > 0) {
pipeline.addLast("client-idle-timeout", new IdleStateHandler(socketTimeoutMillis / 1000,
socketTimeoutMillis / 1000, socketTimeoutMillis / 1000));
}
if (nettyHttpClientConfig.getHttpProxyHandler() != null) {
pipeline.addLast("client-http-proxy", nettyHttpClientConfig.getHttpProxyHandler());
}
if (nettyHttpClientConfig.getSocks4ProxyHandler() != null) {
pipeline.addLast("client-socks4-proxy", nettyHttpClientConfig.getSocks4ProxyHandler());
}
if (nettyHttpClientConfig.getSocks5ProxyHandler() != null) {
Socks5ProxyHandler socks5ProxyHandler = nettyHttpClientConfig.getSocks5ProxyHandler();
pipeline.addLast("client-socks5-proxy", socks5ProxyHandler);
}
configureEncrypted(channel, httpAddress, nettyHttpClient, interaction);
if (nettyCustomizer != null) {
nettyCustomizer.afterChannelInitialized(channel);
}
if (nettyHttpClientConfig.isDebug()) {
logger.log(Level.FINE, "HTTP 1.1 secure channel initialized: " +
" address=" + httpAddress +
" pipeline=" + pipeline.names());
}
}
private void configureEncrypted(Channel channel,
HttpAddress httpAddress,
NettyHttpClient nettyHttpClient,
Interaction interaction) throws IOException {
NettyHttpsClientConfig nettyHttpClientConfig = (NettyHttpsClientConfig) nettyHttpClient.getClientConfig();
ChannelPipeline pipeline = channel.pipeline();
try {
SslHandler sslHandler = createSslHandler(nettyHttpClientConfig, httpAddress);
channel.attr(NettyHttpsClientConfig.ATTRIBUTE_KEY_SSL_HANDLER).set(sslHandler);
pipeline.addLast("client-ssl-handler", sslHandler);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
if (nettyHttpClientConfig.isProtocolNegotiationEnabled()) {
ApplicationProtocolNegotiationHandler negotiationHandler =
new ApplicationProtocolNegotiationHandler(ApplicationProtocolNames.HTTP_1_1) {
@Override
protected void configurePipeline(ChannelHandlerContext ctx, String protocol) throws IOException {
logger.log(Level.FINEST, "configuring pipeline for negotiated protocol " + protocol);
if (ApplicationProtocolNames.HTTP_2.equals(protocol)) {
configureHttp2(ctx.channel(), httpAddress, nettyHttpClient, interaction);
return;
}
if (ApplicationProtocolNames.HTTP_1_1.equals(protocol)) {
configurePlain(ctx.channel(), nettyHttpClient, interaction);
return;
}
ctx.close();
throw new IllegalStateException("protocol not accepted: " + protocol);
}
};
pipeline.addLast("client-negotiation", negotiationHandler);
} else {
configurePlain(channel, nettyHttpClient, interaction);
}
}
private SslHandler createSslHandler(NettyHttpsClientConfig nettyHttpClientConfig,
HttpAddress httpAddress) throws IOException {
SslContextBuilder sslContextBuilder = SslContextBuilder.forClient();
ClientSecureSocketProvider clientSecureSocketProvider = null;
for (ClientSecureSocketProvider provider : ServiceLoader.load(ClientSecureSocketProvider.class)) {
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "trying secure socket provider = " + provider.name());
}
if (nettyHttpClientConfig.getSecureSocketProviderName().equals(provider.name())) {
sslContextBuilder.sslProvider(provider.sslProvider(httpAddress))
.ciphers(provider.ciphers(httpAddress), provider.cipherSuiteFilter(httpAddress));
if (nettyHttpClientConfig.isProtocolNegotiationEnabled()) {
sslContextBuilder.applicationProtocolConfig(new ApplicationProtocolConfig(ApplicationProtocolConfig.Protocol.ALPN,
ApplicationProtocolConfig.SelectorFailureBehavior.NO_ADVERTISE,
ApplicationProtocolConfig.SelectedListenerFailureBehavior.ACCEPT,
ApplicationProtocolNames.HTTP_2, ApplicationProtocolNames.HTTP_1_1));
}
if (provider.securityProvider(httpAddress) != null) {
Provider p = provider.securityProvider(httpAddress);
sslContextBuilder.sslContextProvider(p);
}
if (nettyHttpClientConfig.getTrustManagerFactory() != null) {
sslContextBuilder.trustManager(nettyHttpClientConfig.getTrustManagerFactory());
}
clientSecureSocketProvider = provider;
}
}
InetSocketAddress peer = httpAddress.getInetSocketAddress();
SslHandler sslHandler = sslContextBuilder.build()
.newHandler(nettyHttpClientConfig.getByteBufAllocator(), peer.getHostName(), peer.getPort());
SSLEngine engine = sslHandler.engine();
SSLParameters params = engine.getSSLParameters();
params.setEndpointIdentificationAlgorithm("HTTPS");
List<SNIServerName> sniServerNames = new ArrayList<>();
sniServerNames.add(new SNIHostName(httpAddress.getHost())); // only single host_name allowed
params.setServerNames(sniServerNames);
engine.setSSLParameters(params);
switch (nettyHttpClientConfig.getClientAuthMode()) {
case NEED:
engine.setNeedClientAuth(true);
break;
case WANT:
engine.setWantClientAuth(true);
break;
default:
break;
}
if (clientSecureSocketProvider != null) {
engine.setEnabledProtocols(clientSecureSocketProvider.protocols(httpAddress));
}
if (nettyHttpClientConfig.getSecureProtocolName() != null) {
String[] enabledProtocols = nettyHttpClientConfig.getSecureProtocolName();
engine.setEnabledProtocols(enabledProtocols);
logger.log(Level.FINEST, "TLS: configured protocol = " +
Arrays.asList(nettyHttpClientConfig.getSecureProtocolName()));
}
sslHandler.setHandshakeTimeoutMillis(nettyHttpClientConfig.getSocketConfig().getSslHandshakeTimeoutMillis());
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "TLS: selected secure socket provider = " +
(clientSecureSocketProvider != null ? clientSecureSocketProvider.name() : "<none>"));
logger.log(Level.FINEST, "TLS:" +
" enabled protocols = " + Arrays.asList(engine.getEnabledProtocols()) +
" supported protocols = " + Arrays.asList(engine.getSupportedProtocols()) +
" application protocol = " + engine.getApplicationProtocol() +
" handshake application protocol = " + engine.getHandshakeApplicationProtocol());
logger.log(Level.FINEST, "TLS: client need auth = " +
engine.getNeedClientAuth() + " client want auth = " + engine.getWantClientAuth());
}
return sslHandler;
}
private void configureHttp2(Channel channel,
HttpAddress httpAddress,
NettyHttpClient nettyHttpClient,
Interaction interaction) throws IOException {
NettyHttpClientConfig nettyHttpClientConfig = nettyHttpClient.getClientConfig();
ChannelPipeline pipeline = channel.pipeline();
ChannelInitializer<Channel> initializer = new ChannelInitializer<>() {
@Override
protected void initChannel(Channel ch) {
throw new IllegalStateException();
}
};
Http2MultiplexCodecBuilder multiplexCodecBuilder = Http2MultiplexCodecBuilder.forClient(initializer)
.initialSettings(nettyHttpClientConfig.getHttp2Settings());
if (nettyHttpClientConfig.isDebug()) {
multiplexCodecBuilder.frameLogger(new Http2FrameLogger(LogLevel.DEBUG, "client-frame"));
}
Http2MultiplexCodec multiplexCodec = multiplexCodecBuilder.autoAckSettingsFrame(true).build();
pipeline.addLast("client-multiplex", multiplexCodec);
pipeline.addLast("client-messages", new Http2Messages(interaction));
// simulate we are ready for HTTP/2
interaction.settingsReceived(Http2Settings.defaultSettings());
}
private void configurePlain(Channel channel,
NettyHttpClient nettyHttpClient,
Interaction interaction) throws IOException {
NettyHttpClientConfig nettyHttpClientConfig = nettyHttpClient.getClientConfig();
ChannelPipeline pipeline = channel.pipeline();
pipeline.addLast("http-client-chunk-writer",
new ChunkedWriteHandler());
pipeline.addLast("http-client-codec", new HttpClientCodec(nettyHttpClientConfig.getMaxInitialLineLength(),
nettyHttpClientConfig.getMaxHeadersSize(), nettyHttpClientConfig.getMaxChunkSize()));
if (nettyHttpClientConfig.isGzipEnabled()) {
pipeline.addLast("http-client-decompressor", new HttpContentDecompressor());
}
HttpObjectAggregator httpObjectAggregator =
new HttpObjectAggregator(nettyHttpClientConfig.getMaxContentLength(), false);
httpObjectAggregator.setMaxCumulationBufferComponents(nettyHttpClientConfig.getMaxCompositeBufferComponents());
pipeline.addLast("http-client-aggregator", httpObjectAggregator);
pipeline.addLast("http-client-response", new Http1Handler(interaction));
interaction.settingsReceived(null);
}
}

View file

@ -0,0 +1,32 @@
package org.xbib.net.http.client.netty.secure.http1;
import io.netty.channel.Channel;
import io.netty.handler.ssl.SslHandler;
import javax.net.ssl.SSLSession;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.client.netty.HttpResponseBuilder;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.http1.Http1Interaction;
import org.xbib.net.http.client.netty.http2.Http2Interaction;
import org.xbib.net.http.client.netty.secure.HttpsResponse;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
import org.xbib.net.http.client.netty.secure.http2.Https2Interaction;
public class Https1Interaction extends Http1Interaction {
public Https1Interaction(NettyHttpClient nettyHttpClient, HttpAddress httpAddress) {
super(nettyHttpClient, httpAddress);
}
@Override
protected HttpResponseBuilder newHttpResponseBuilder(Channel channel) {
SslHandler sslHandler = channel.attr(NettyHttpsClientConfig.ATTRIBUTE_KEY_SSL_HANDLER).get();
SSLSession sslSession = sslHandler != null ? sslHandler.engine().getSession() : null;
return HttpsResponse.builder().setSSLSession(sslSession);
}
@Override
protected Http2Interaction upgradeInteraction() {
return new Https2Interaction(nettyHttpClient, httpAddress);
}
}

View file

@ -0,0 +1,187 @@
package org.xbib.net.http.client.netty.secure.http2;
import io.netty.channel.Channel;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.handler.codec.http2.Http2FrameLogger;
import io.netty.handler.codec.http2.Http2MultiplexCodec;
import io.netty.handler.codec.http2.Http2MultiplexCodecBuilder;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.ssl.ApplicationProtocolConfig;
import io.netty.handler.ssl.ApplicationProtocolNames;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.SslHandler;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.net.InetSocketAddress;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.ServiceLoader;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.net.ssl.SNIHostName;
import javax.net.ssl.SNIServerName;
import javax.net.ssl.SSLEngine;
import javax.net.ssl.SSLParameters;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpVersion;
import org.xbib.net.http.client.netty.HttpChannelInitializer;
import org.xbib.net.http.client.netty.Interaction;
import org.xbib.net.http.client.netty.NettyCustomizer;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.http2.Http2Messages;
import org.xbib.net.http.client.netty.TrafficLoggingHandler;
import org.xbib.net.http.client.netty.secure.ClientSecureSocketProvider;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
public class Https2ChannelInitializer implements HttpChannelInitializer {
private static final Logger logger = Logger.getLogger(Https2ChannelInitializer.class.getName());
public Https2ChannelInitializer() {
}
@Override
public boolean supports(HttpAddress address) {
return HttpVersion.HTTP_2_0.equals(address.getVersion()) && address.isSecure();
}
@Override
public Interaction newInteraction(NettyHttpClient client, HttpAddress httpAddress) {
return new Https2Interaction(client, httpAddress);
}
@Override
public void init(Channel channel,
HttpAddress httpAddress,
NettyHttpClient nettyHttpClient,
NettyCustomizer nettyCustomizer,
Interaction interaction) {
NettyHttpClientConfig nettyHttpClientConfig = nettyHttpClient.getClientConfig();
if (nettyHttpClientConfig.isDebug()) {
channel.pipeline().addLast(new TrafficLoggingHandler(LogLevel.DEBUG));
}
configureEncrypted(channel, httpAddress, nettyHttpClient, interaction);
if (nettyCustomizer != null) {
nettyCustomizer.afterChannelInitialized(channel);
}
if (nettyHttpClientConfig.isDebug()) {
logger.log(Level.FINE, "HTTP/2 secure channel initialized: address = " + httpAddress +
" pipeline = " + channel.pipeline().names());
}
}
private void configureEncrypted(Channel channel,
HttpAddress httpAddress,
NettyHttpClient nettyHttpClient,
Interaction interaction) {
NettyHttpsClientConfig nettyHttpClientConfig = (NettyHttpsClientConfig) nettyHttpClient.getClientConfig();
try {
SslHandler sslHandler = createSslHandler(nettyHttpClientConfig, httpAddress);
channel.attr(NettyHttpsClientConfig.ATTRIBUTE_KEY_SSL_HANDLER).set(sslHandler);
channel.pipeline().addLast("client-ssl-handler", sslHandler);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
configurePlain(channel, nettyHttpClient, interaction);
}
private SslHandler createSslHandler(NettyHttpsClientConfig nettyHttpClientConfig,
HttpAddress httpAddress) throws IOException {
SslContextBuilder sslContextBuilder = SslContextBuilder.forClient();
ClientSecureSocketProvider clientSecureSocketProvider = null;
for (ClientSecureSocketProvider provider : ServiceLoader.load(ClientSecureSocketProvider.class)) {
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "trying secure socket provider = " + provider.name());
}
if (nettyHttpClientConfig.getSecureSocketProviderName().equals(provider.name())) {
sslContextBuilder.sslProvider(provider.sslProvider(httpAddress))
.ciphers(provider.ciphers(httpAddress), provider.cipherSuiteFilter(httpAddress));
sslContextBuilder.applicationProtocolConfig(new ApplicationProtocolConfig(ApplicationProtocolConfig.Protocol.ALPN,
ApplicationProtocolConfig.SelectorFailureBehavior.NO_ADVERTISE,
ApplicationProtocolConfig.SelectedListenerFailureBehavior.ACCEPT,
ApplicationProtocolNames.HTTP_2));
if (provider.securityProvider(httpAddress) != null) {
sslContextBuilder.sslContextProvider(provider.securityProvider(httpAddress));
}
if (nettyHttpClientConfig.getTrustManagerFactory() != null) {
sslContextBuilder.trustManager(nettyHttpClientConfig.getTrustManagerFactory());
}
clientSecureSocketProvider = provider;
}
}
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "selected secure socket provider = " +
(clientSecureSocketProvider != null ? clientSecureSocketProvider.name() : "<none>"));
}
InetSocketAddress peer = httpAddress.getInetSocketAddress();
SslHandler sslHandler = sslContextBuilder.build()
.newHandler(nettyHttpClientConfig.getByteBufAllocator(), peer.getHostName(), peer.getPort());
SSLEngine engine = sslHandler.engine();
SSLParameters params = engine.getSSLParameters();
params.setEndpointIdentificationAlgorithm("HTTPS");
List<SNIServerName> sniServerNames = new ArrayList<>();
sniServerNames.add(new SNIHostName(httpAddress.getHost())); // only single host_name allowed
params.setServerNames(sniServerNames);
engine.setSSLParameters(params);
switch (nettyHttpClientConfig.getClientAuthMode()) {
case NEED:
engine.setNeedClientAuth(true);
break;
case WANT:
engine.setWantClientAuth(true);
break;
default:
break;
}
if (clientSecureSocketProvider != null) {
engine.setEnabledProtocols(clientSecureSocketProvider.protocols(httpAddress));
}
if (nettyHttpClientConfig.getSecureProtocolName() != null) {
String[] enabledProtocols = nettyHttpClientConfig.getSecureProtocolName();
engine.setEnabledProtocols(enabledProtocols);
logger.log(Level.FINEST, "TLS: configured protocol = " +
Arrays.asList(nettyHttpClientConfig.getSecureProtocolName()));
}
sslHandler.setHandshakeTimeoutMillis(nettyHttpClientConfig.getSocketConfig().getSslHandshakeTimeoutMillis());
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "TLS: selected secure socket provider = " +
(clientSecureSocketProvider != null ? clientSecureSocketProvider.name() : "<none>"));
logger.log(Level.FINEST, "TLS: " +
" enabled protocols = " + Arrays.asList(engine.getEnabledProtocols()) +
" supported protocols = " + Arrays.asList(engine.getSupportedProtocols()) +
" application protocol = " + engine.getApplicationProtocol() +
" handshake application protocol = " + engine.getHandshakeApplicationProtocol());
logger.log(Level.FINEST, "TLS: client need auth = " +
engine.getNeedClientAuth() + " client want auth = " + engine.getWantClientAuth());
}
return sslHandler;
}
private void configurePlain(Channel channel,
NettyHttpClient nettyHttpClient,
Interaction interaction) {
NettyHttpClientConfig nettyHttpClientConfig = nettyHttpClient.getClientConfig();
ChannelInitializer<Channel> initializer = new ChannelInitializer<>() {
@Override
protected void initChannel(Channel ch) {
throw new IllegalStateException();
}
};
Http2MultiplexCodecBuilder multiplexCodecBuilder = Http2MultiplexCodecBuilder.forClient(initializer)
.initialSettings(nettyHttpClientConfig.getHttp2Settings());
if (nettyHttpClientConfig.isDebug()) {
multiplexCodecBuilder.frameLogger(new Http2FrameLogger(LogLevel.DEBUG, "client-frame"));
}
Http2MultiplexCodec multiplexCodec = multiplexCodecBuilder
.autoAckSettingsFrame(true)
.autoAckPingFrame(true)
.gracefulShutdownTimeoutMillis(30000L)
.build();
ChannelPipeline pipeline = channel.pipeline();
pipeline.addLast("client-multiplex", multiplexCodec);
pipeline.addLast("client-messages", new Http2Messages(interaction));
}
}

View file

@ -0,0 +1,29 @@
package org.xbib.net.http.client.netty.secure.http2;
import io.netty.channel.Channel;
import io.netty.handler.ssl.SslHandler;
import org.xbib.net.http.client.netty.Interaction;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.http2.Http2ChildChannelInitializer;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
public class Https2ChildChannelInitializer extends Http2ChildChannelInitializer {
public Https2ChildChannelInitializer(NettyHttpClientConfig clientConfig, Interaction interaction, Channel parentChannel) {
super(clientConfig, interaction, parentChannel);
}
/**
* Initialize child channel for HTTP/2, copy the SSL handler attribute so it can be found in interactions.
*
* @param ch the {@link Channel} which was registered.
*/
@Override
protected void initChannel(Channel ch) {
super.initChannel(ch);
SslHandler sslHandler = parentChannel.attr(NettyHttpsClientConfig.ATTRIBUTE_KEY_SSL_HANDLER).get();
if (sslHandler != null) {
ch.attr(NettyHttpsClientConfig.ATTRIBUTE_KEY_SSL_HANDLER).set(sslHandler);
}
}
}

View file

@ -0,0 +1,37 @@
package org.xbib.net.http.client.netty.secure.http2;
import io.netty.channel.Channel;
import io.netty.handler.ssl.SslHandler;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.client.netty.HttpResponseBuilder;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.http2.Http2ChildChannelInitializer;
import org.xbib.net.http.client.netty.http2.Http2Interaction;
import org.xbib.net.http.client.netty.secure.HttpsResponse;
import org.xbib.net.http.client.netty.secure.HttpsResponseBuilder;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
public class Https2Interaction extends Http2Interaction {
public Https2Interaction(NettyHttpClient nettyHttpClient, HttpAddress httpAddress) {
super(nettyHttpClient, httpAddress);
}
@Override
protected Http2ChildChannelInitializer newHttp2ChildChannelInitializer(NettyHttpClientConfig clientConfig,
Http2Interaction interaction,
Channel parentChannel) {
return new Https2ChildChannelInitializer(clientConfig, interaction, parentChannel);
}
@Override
protected HttpResponseBuilder newHttpResponseBuilder(Channel channel) {
SslHandler sslHandler = channel.attr(NettyHttpsClientConfig.ATTRIBUTE_KEY_SSL_HANDLER).get();
HttpsResponseBuilder builder = HttpsResponse.builder();
if (sslHandler != null) {
builder.setSSLSession(sslHandler.engine().getSession());
}
return builder;
}
}

View file

@ -0,0 +1,2 @@
org.xbib.net.http.client.netty.secure.http1.Https1ChannelInitializer
org.xbib.net.http.client.netty.secure.http2.Https2ChannelInitializer

View file

@ -0,0 +1 @@
org.xbib.net.http.client.netty.secure.JdkClientSecureSocketProvider

View file

@ -0,0 +1,39 @@
package org.xbib.net.http.netty.client.secure;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.junit.jupiter.api.Test;
import org.xbib.net.http.client.netty.HttpRequest;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
public class AkamaiTest {
private static final Logger logger = Logger.getLogger(AkamaiTest.class.getName());
/**
* Problems with akamai:
* failing: Cannot invoke "io.netty.handler.codec.http2.AbstractHttp2StreamChannel.fireChildRead(io.netty.handler.codec.http2.Http2Frame)" because "channel" is null * demo/h2_demo_frame.html sends no content, only a push promise, and does not continue
*
* @throws IOException if test fails
*/
@Test
void testAkamai() throws IOException {
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("https://http2.akamai.com/demo/h2_demo_frame.html")
.setVersion("HTTP/2.0")
.setResponseListener(resp -> logger.log(Level.INFO, "got HTTP/2 response: " +
resp.getHeaders() + resp.getBodyAsChars(StandardCharsets.UTF_8)))
.build();
client.execute(request).get().close();
}
}
}

View file

@ -0,0 +1,155 @@
package org.xbib.net.http.netty.client.secure;
import org.junit.jupiter.api.Test;
import org.xbib.net.http.client.BackOff;
import org.xbib.net.http.client.ExponentialBackOff;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* Tests {@link ExponentialBackOff}.
*/
class ExponentialBackOffTest {
@Test
void testConstructor() {
ExponentialBackOff backOffPolicy = new ExponentialBackOff();
assertEquals(ExponentialBackOff.DEFAULT_INITIAL_INTERVAL_MILLIS,
backOffPolicy.getInitialIntervalMillis());
assertEquals(ExponentialBackOff.DEFAULT_INITIAL_INTERVAL_MILLIS,
backOffPolicy.getCurrentIntervalMillis());
assertEquals(ExponentialBackOff.DEFAULT_RANDOMIZATION_FACTOR,
backOffPolicy.getRandomizationFactor(), 1);
assertEquals(ExponentialBackOff.DEFAULT_MULTIPLIER, backOffPolicy.getMultiplier(), 1);
assertEquals(
ExponentialBackOff.DEFAULT_MAX_INTERVAL_MILLIS, backOffPolicy.getMaxIntervalMillis());
assertEquals(ExponentialBackOff.DEFAULT_MAX_ELAPSED_TIME_MILLIS,
backOffPolicy.getMaxElapsedTimeMillis());
}
@Test
void testBuilder() {
ExponentialBackOff backOffPolicy = new ExponentialBackOff.Builder().build();
assertEquals(ExponentialBackOff.DEFAULT_INITIAL_INTERVAL_MILLIS,
backOffPolicy.getInitialIntervalMillis());
assertEquals(ExponentialBackOff.DEFAULT_INITIAL_INTERVAL_MILLIS,
backOffPolicy.getCurrentIntervalMillis());
assertEquals(ExponentialBackOff.DEFAULT_RANDOMIZATION_FACTOR,
backOffPolicy.getRandomizationFactor(), 1);
assertEquals(ExponentialBackOff.DEFAULT_MULTIPLIER, backOffPolicy.getMultiplier(), 1);
assertEquals(ExponentialBackOff.DEFAULT_MAX_INTERVAL_MILLIS, backOffPolicy.getMaxIntervalMillis());
assertEquals(ExponentialBackOff.DEFAULT_MAX_ELAPSED_TIME_MILLIS,
backOffPolicy.getMaxElapsedTimeMillis());
int testInitialInterval = 1;
double testRandomizationFactor = 0.1;
double testMultiplier = 5.0;
int testMaxInterval = 10;
int testMaxElapsedTime = 900000;
backOffPolicy = new ExponentialBackOff.Builder()
.setInitialIntervalMillis(testInitialInterval)
.setRandomizationFactor(testRandomizationFactor)
.setMultiplier(testMultiplier)
.setMaxIntervalMillis(testMaxInterval)
.setMaxElapsedTimeMillis(testMaxElapsedTime)
.build();
assertEquals(testInitialInterval, backOffPolicy.getInitialIntervalMillis());
assertEquals(testInitialInterval, backOffPolicy.getCurrentIntervalMillis());
assertEquals(testRandomizationFactor, backOffPolicy.getRandomizationFactor(), 1);
assertEquals(testMultiplier, backOffPolicy.getMultiplier(), 1);
assertEquals(testMaxInterval, backOffPolicy.getMaxIntervalMillis());
assertEquals(testMaxElapsedTime, backOffPolicy.getMaxElapsedTimeMillis());
}
@Test
void testBackOff() {
int testInitialInterval = 500;
double testRandomizationFactor = 0.1;
double testMultiplier = 2.0;
int testMaxInterval = 5000;
int testMaxElapsedTime = 900000;
ExponentialBackOff backOffPolicy = new ExponentialBackOff.Builder()
.setInitialIntervalMillis(testInitialInterval)
.setRandomizationFactor(testRandomizationFactor)
.setMultiplier(testMultiplier)
.setMaxIntervalMillis(testMaxInterval)
.setMaxElapsedTimeMillis(testMaxElapsedTime)
.build();
int[] expectedResults = {500, 1000, 2000, 4000, 5000, 5000, 5000, 5000, 5000, 5000};
for (int expected : expectedResults) {
assertEquals(expected, backOffPolicy.getCurrentIntervalMillis());
// Assert that the next back off falls in the expected range.
int minInterval = (int) (expected - (testRandomizationFactor * expected));
int maxInterval = (int) (expected + (testRandomizationFactor * expected));
long actualInterval = backOffPolicy.nextBackOffMillis();
assertTrue(minInterval <= actualInterval && actualInterval <= maxInterval);
}
}
@Test
void testGetRandomizedInterval() {
// 33% chance of being 1.
assertEquals(1, ExponentialBackOff.getRandomValueFromInterval(0.5, 0, 2));
assertEquals(1, ExponentialBackOff.getRandomValueFromInterval(0.5, 0.33, 2));
// 33% chance of being 2.
assertEquals(2, ExponentialBackOff.getRandomValueFromInterval(0.5, 0.34, 2));
assertEquals(2, ExponentialBackOff.getRandomValueFromInterval(0.5, 0.66, 2));
// 33% chance of being 3.
assertEquals(3, ExponentialBackOff.getRandomValueFromInterval(0.5, 0.67, 2));
assertEquals(3, ExponentialBackOff.getRandomValueFromInterval(0.5, 0.99, 2));
}
@Test
void testGetElapsedTimeMillis() {
ExponentialBackOff backOffPolicy = new ExponentialBackOff.Builder().setNanoClock(new MyNanoClock()).build();
long elapsedTimeMillis = backOffPolicy.getElapsedTimeMillis();
assertEquals(1000, elapsedTimeMillis);
}
@Test
void testMaxElapsedTime() {
ExponentialBackOff backOffPolicy =
new ExponentialBackOff.Builder().setNanoClock(new MyNanoClock(10000)).build();
assertTrue(backOffPolicy.nextBackOffMillis() != BackOff.STOP);
// Change the currentElapsedTimeMillis to be 0 ensuring that the elapsed time will be greater
// than the max elapsed time.
backOffPolicy.setStartTimeNanos(0);
assertEquals(BackOff.STOP, backOffPolicy.nextBackOffMillis());
}
@Test
void testBackOffOverflow() {
int testInitialInterval = Integer.MAX_VALUE / 2;
double testMultiplier = 2.1;
int testMaxInterval = Integer.MAX_VALUE;
ExponentialBackOff backOffPolicy = new ExponentialBackOff.Builder()
.setInitialIntervalMillis(testInitialInterval)
.setMultiplier(testMultiplier)
.setMaxIntervalMillis(testMaxInterval)
.build();
backOffPolicy.nextBackOffMillis();
// Assert that when an overflow is possible the current interval is set to the max interval.
assertEquals(testMaxInterval, backOffPolicy.getCurrentIntervalMillis());
}
static class MyNanoClock implements ExponentialBackOff.NanoClock {
private int i = 0;
private long startSeconds;
MyNanoClock() {
}
MyNanoClock(long startSeconds) {
this.startSeconds = startSeconds;
}
public long nanoTime() {
return (startSeconds + i++) * 1000000000;
}
}
}

View file

@ -0,0 +1,43 @@
package org.xbib.net.http.netty.client.secure;
import com.sun.management.UnixOperatingSystemMXBean;
import java.lang.management.ManagementFactory;
import java.lang.management.OperatingSystemMXBean;
import java.nio.charset.StandardCharsets;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.junit.jupiter.api.Test;
import org.xbib.net.http.client.netty.HttpRequest;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
class FileDescriptorLeakTest {
private static final Logger logger = Logger.getLogger(FileDescriptorLeakTest.class.getName());
@Test
void testFileLeak() throws Exception {
OperatingSystemMXBean os = ManagementFactory.getOperatingSystemMXBean();
for (int i = 0; i < 3; i++) {
if (os instanceof UnixOperatingSystemMXBean) {
logger.info("before: number of open file descriptor : " + ((UnixOperatingSystemMXBean) os).getOpenFileDescriptorCount());
}
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("https://xbib.org")
.setResponseListener(resp -> logger.log(Level.INFO, "got response: " +
resp.getHeaders() + resp.getBodyAsChars(StandardCharsets.UTF_8)))
.build();
client.execute(request).get().close();
}
if (os instanceof UnixOperatingSystemMXBean){
logger.info("after: number of open file descriptor : " + ((UnixOperatingSystemMXBean) os).getOpenFileDescriptorCount());
}
}
}
}

View file

@ -0,0 +1,117 @@
package org.xbib.net.http.netty.client.secure;
import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.channel.Channel;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.http.HttpMethod;
import io.netty.handler.codec.http2.DefaultHttp2Headers;
import io.netty.handler.codec.http2.Http2ConnectionHandler;
import io.netty.handler.codec.http2.Http2ConnectionHandlerBuilder;
import io.netty.handler.codec.http2.Http2Exception;
import io.netty.handler.codec.http2.Http2FrameAdapter;
import io.netty.handler.codec.http2.Http2FrameLogger;
import io.netty.handler.codec.http2.Http2SecurityUtil;
import io.netty.handler.codec.http2.Http2Settings;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.ssl.ApplicationProtocolConfig;
import io.netty.handler.ssl.ApplicationProtocolNames;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.SslHandler;
import io.netty.handler.ssl.SslProvider;
import io.netty.handler.ssl.SupportedCipherSuiteFilter;
import io.netty.handler.ssl.util.InsecureTrustManagerFactory;
import java.net.InetSocketAddress;
import java.util.Arrays;
import java.util.concurrent.CompletableFuture;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.net.ssl.SNIHostName;
import javax.net.ssl.SNIServerName;
import javax.net.ssl.SSLEngine;
import javax.net.ssl.SSLParameters;
import org.junit.jupiter.api.Test;
class Http2FramesTest {
private static final Logger logger = Logger.getLogger(Http2FramesTest.class.getName());
@Test
void testHttp2Frames() throws Exception {
final InetSocketAddress inetSocketAddress = new InetSocketAddress("webtide.com", 443);
CompletableFuture<Boolean> completableFuture = new CompletableFuture<>();
EventLoopGroup eventLoopGroup = new NioEventLoopGroup();
Channel clientChannel = null;
try {
Bootstrap bootstrap = new Bootstrap()
.group(eventLoopGroup)
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<Channel>() {
@Override
protected void initChannel(Channel ch) throws Exception {
SslContext sslContext = SslContextBuilder.forClient()
.sslProvider(SslProvider.JDK)
.trustManager(InsecureTrustManagerFactory.INSTANCE)
.ciphers(Http2SecurityUtil.CIPHERS, SupportedCipherSuiteFilter.INSTANCE)
.applicationProtocolConfig(new ApplicationProtocolConfig(
ApplicationProtocolConfig.Protocol.ALPN,
ApplicationProtocolConfig.SelectorFailureBehavior.NO_ADVERTISE,
ApplicationProtocolConfig.SelectedListenerFailureBehavior.ACCEPT,
ApplicationProtocolNames.HTTP_2))
.build();
SslHandler sslHandler = sslContext.newHandler(ch.alloc());
SSLEngine engine = sslHandler.engine();
String fullQualifiedHostname = inetSocketAddress.getHostName();
SSLParameters params = engine.getSSLParameters();
params.setServerNames(Arrays.asList(new SNIServerName[]{new SNIHostName(fullQualifiedHostname)}));
engine.setSSLParameters(params);
ch.pipeline().addLast(sslHandler);
Http2FrameAdapter frameAdapter = new Http2FrameAdapter() {
@Override
public void onSettingsRead(ChannelHandlerContext ctx, Http2Settings settings) {
logger.log(Level.FINE, "settings received, now writing request");
Http2ConnectionHandler handler = ctx.pipeline().get(Http2ConnectionHandler.class);
handler.encoder().writeHeaders(ctx, 3,
new DefaultHttp2Headers().method(HttpMethod.GET.asciiName())
.path("/")
.scheme("https")
.authority(inetSocketAddress.getHostName()),
0, true, ctx.newPromise());
ctx.channel().flush();
}
@Override
public int onDataRead(ChannelHandlerContext ctx, int streamId, ByteBuf data, int padding,
boolean endOfStream) throws Http2Exception {
int i = super.onDataRead(ctx, streamId, data, padding, endOfStream);
if (endOfStream) {
completableFuture.complete(true);
}
return i;
}
};
Http2ConnectionHandlerBuilder builder = new Http2ConnectionHandlerBuilder()
.server(false)
.frameListener(frameAdapter)
.frameLogger(new Http2FrameLogger(LogLevel.INFO, "client"));
ch.pipeline().addLast(builder.build());
}
});
logger.log(Level.INFO, () -> "connecting");
clientChannel = bootstrap.connect(inetSocketAddress).sync().channel();
logger.log(Level.INFO, () -> "waiting for end of stream");
completableFuture.get();
logger.log(Level.INFO, () -> "done");
} finally {
if (clientChannel != null) {
clientChannel.close();
}
eventLoopGroup.shutdownGracefully();
}
}
}

View file

@ -0,0 +1,58 @@
package org.xbib.net.http.netty.client.secure;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.junit.jupiter.api.Test;
import org.xbib.net.http.client.netty.HttpRequest;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
import org.xbib.net.http.cookie.Cookie;
import static org.junit.jupiter.api.Assertions.assertTrue;
class HttpBinTest {
private static final Logger logger = Logger.getLogger(HttpBinTest.class.getName());
/**
* Test httpbin.org "Set-Cookie:" header after redirection of URL.
*
* The reponse body should be
* <pre>
* {
* "cookies": {
* "name": "value"
* }
* }
* </pre>
* @throws IOException if test fails
*/
@Test
void testHttpBinCookies() throws IOException {
AtomicBoolean success = new AtomicBoolean();
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("http://httpbin.org/cookies/set?name=value")
.setResponseListener(resp -> {
logger.log(Level.INFO, "got HTTP/2 response: " +
resp.getHeaders() + resp.getBodyAsChars(StandardCharsets.UTF_8));
for (Cookie cookie : resp.getCookies()) {
logger.log(Level.INFO, "got cookie: " + cookie.toString());
if ("name".equals(cookie.name()) && ("value".equals(cookie.value()))) {
success.set(true);
}
}
})
.build();
client.execute(request).get().close();
}
assertTrue(success.get());
}
}

View file

@ -0,0 +1,295 @@
package org.xbib.net.http.netty.client.secure;
import io.netty.handler.proxy.Socks5ProxyHandler;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.security.cert.Certificate;
import java.security.cert.CertificateParsingException;
import java.security.cert.X509Certificate;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Function;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.net.ssl.SSLPeerUnverifiedException;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
import org.xbib.net.SocketConfig;
import org.xbib.net.http.HttpMethod;
import org.xbib.net.http.client.HttpResponse;
import org.xbib.net.http.client.netty.HttpRequest;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.secure.HttpsResponse;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
import static org.junit.jupiter.api.Assertions.assertEquals;
class Https1Test {
private static final Logger logger = Logger.getLogger(Https1Test.class.getName());
@Test
void testXbib() throws Exception {
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("https://xbib.org")
.setResponseListener(resp ->
logger.log(Level.INFO,
"got response: " +
" status = " + resp.getStatus() +
" headers = " + resp.getHeaders() +
" body = " + resp.getBodyAsChars(StandardCharsets.UTF_8) +
" ssl = " + dumpCertificates((HttpsResponse) resp)))
.build();
client.execute(request).get().close();
}
}
@Test
void testGoogleHttp() throws Exception {
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setProtocolNegotiation(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("http://google.de")
.setResponseListener(resp ->
logger.log(Level.INFO,
"got response: " +
" status = " + resp.getStatus() +
" headers = " + resp.getHeaders() +
" body = " + resp.getBodyAsChars(StandardCharsets.UTF_8)))
.build();
client.execute(request).get().close();
}
}
@Test
void testGoogleUpgradeHttps() throws Exception {
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setProtocolNegotiation(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("https://www.google.de/")
.setResponseListener(resp ->
logger.log(Level.INFO,
"got response: " +
" status = " + resp.getStatus() +
" headers = " + resp.getHeaders() +
" body = " + resp.getBodyAsChars(StandardCharsets.UTF_8) +
" ssl = " + dumpCertificates((HttpsResponse) resp)))
.build();
client.execute(request).get().close();
}
}
@Test
void testDNB() throws Exception {
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
Map<String, Object> map = Map.of(
"version", "1.1",
"operation", "searchRetrieve",
"recordSchema", "MARC21plus-1-xml",
"query", "iss=00280836"
);
HttpRequest request = HttpRequest.get()
.setURL("http://services.dnb.de/sru/zdb")
.setParameters(map)
.setResponseListener(resp -> logger.log(Level.INFO,
"got response: " + resp.getHeaders() +
resp.getBodyAsChars(StandardCharsets.UTF_8) +
" status=" + resp.getStatus()))
.build();
client.execute(request).get().close();
}
}
@Test
void testHebisGetRequest() throws Exception {
// we test HEBIS here with strange certificate setup and TLS 1.2 only
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()){
HttpRequest request = HttpRequest.post()
.setURL("https://hebis.rz.uni-frankfurt.de/HEBCGI/vuefl_recv_data.pl")
.setResponseListener(resp ->
logger.log(Level.INFO,
"got response: " +
" status = " + resp.getStatus() +
" headers = " + resp.getHeaders() +
" body = " + resp.getBodyAsChars(StandardCharsets.UTF_8) +
" ssl = " + dumpCertificates((HttpsResponse) resp))
)
.build();
client.execute(request).get().close();
}
}
@Test
void testSequentialRequests() throws Exception {
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
for (int i = 0; i <10; i++) {
HttpRequest request = HttpRequest.get().setURL("https://xbib.org")
.setResponseListener(resp ->
logger.log(Level.INFO,
"got response: " +
" status = " + resp.getStatus() +
" headers = " + resp.getHeaders() +
" body = " + resp.getBodyAsChars(StandardCharsets.UTF_8) +
" ssl = " + dumpCertificates((HttpsResponse) resp)))
.build();
client.execute(request).get();
}
}
}
@Test
void testParallelRequests() throws Exception {
AtomicInteger counter = new AtomicInteger();
NettyHttpClientConfig config = new NettyHttpsClientConfig();
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request1 = HttpRequest.builder(HttpMethod.GET)
.setURL("https://xbib.org")
.setVersion("HTTP/1.1")
.setResponseListener(resp ->
logger.log(Level.INFO,
"got response: " +
" counter = " + counter.incrementAndGet() +
" status = " + resp.getStatus() +
" headers = " + resp.getHeaders() +
" body = " + resp.getBodyAsChars(StandardCharsets.UTF_8) +
" ssl = " + dumpCertificates((HttpsResponse) resp)))
.build();
HttpRequest request2 = HttpRequest.builder(HttpMethod.GET)
.setURL("https://xbib.org")
.setVersion("HTTP/1.1")
.setResponseListener(resp ->
logger.log(Level.INFO,
"got response: " +
" counter = " + counter.incrementAndGet() +
" status = " + resp.getStatus() +
" headers = " + resp.getHeaders() +
" body = " + resp.getBodyAsChars(StandardCharsets.UTF_8) +
" ssl = " + dumpCertificates((HttpsResponse) resp)))
.build();
for (int i = 0; i < 5; i++) {
client.execute(request1);
client.execute(request2);
}
Thread.sleep(1000L);
}
assertEquals(10, counter.get());
}
@Test
void testXbibOrgWithCompletableFuture() throws IOException {
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient httpClient = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("https://xbib.org")
.build();
String result = httpClient.execute(request, response -> response.getBodyAsChars(StandardCharsets.UTF_8).toString())
.exceptionally(Throwable::getMessage)
.join();
logger.info("got result = " + result);
}
// TODO 15 sec timeout on closing event loop group, why?
}
@Test
void testXbibOrgWithCompletableFutureAndGoogleSearch() throws IOException {
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient httpClient = NettyHttpClient.builder()
.setConfig(config)
.build()) {
final Function<HttpResponse, String> stringFunction =
response -> response.getBodyAsChars(StandardCharsets.UTF_8).toString();
HttpRequest request = HttpRequest.get()
.setURL("https://xbib.org")
.build();
final CompletableFuture<String> completableFuture = httpClient.execute(request, stringFunction)
.exceptionally(Throwable::getMessage)
.thenCompose(content -> {
try {
return httpClient.execute(HttpRequest.get()
.setURL("https://www.google.de/")
.addParameter("query", content.substring(0, 15))
.build(), stringFunction);
} catch (IOException e) {
logger.log(Level.WARNING, e.getMessage(), e);
return null;
}
});
String result = completableFuture.join();
logger.info("got result = " + result);
}
}
@Disabled("proxy is down")
@Test
void testXbibOrgWithProxy() throws IOException {
SocketConfig socketConfig = new SocketConfig();
socketConfig.setConnectTimeoutMillis(30000);
socketConfig.setReadTimeoutMillis(30000);
Socks5ProxyHandler handler = new Socks5ProxyHandler(new InetSocketAddress("178.162.202.44", 1695));
handler.setConnectTimeoutMillis(30000L);
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setSocketConfig(socketConfig)
.setSocks5ProxyHandler(handler)
.setDebug(true);
try (NettyHttpClient httpClient = NettyHttpClient.builder()
.setConfig(config)
.build()) {
httpClient.execute(HttpRequest.get()
.setURL("https://xbib.org")
.setResponseListener(resp -> logger.log(Level.INFO, "status = " + resp.getStatus() +
" response body = " + resp.getBodyAsChars(StandardCharsets.UTF_8)))
.build())
.get();
}
}
private String dumpCertificates(HttpsResponse httpsResponse) {
StringBuilder sb = new StringBuilder();
try {
for (Certificate certificate : httpsResponse.getSSLSession().getPeerCertificates()) {
if (certificate instanceof X509Certificate) {
X509Certificate c = (X509Certificate) certificate;
sb.append("subjects=").append(c.getSubjectAlternativeNames());
sb.append(",issuers=").append(c.getIssuerAlternativeNames());
sb.append(",not before=").append(c.getNotBefore());
sb.append(",not after=").append(c.getNotAfter());
sb.append("\n");
}
}
} catch (SSLPeerUnverifiedException | CertificateParsingException e) {
logger.log(Level.WARNING, e.getMessage(), e);
}
return sb.toString();
}
}

View file

@ -0,0 +1,116 @@
package org.xbib.net.http.netty.client.secure;
import io.netty.handler.codec.http.HttpMethod;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
import org.xbib.net.http.client.netty.HttpRequest;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
import static org.junit.jupiter.api.Assertions.assertTrue;
public class Https2Test {
private static final Logger logger = Logger.getLogger(Https2Test.class.getName());
@Disabled
@Test
void testXbib() throws Exception {
// the xbib server does not offer HTTP/2 so this does not work!
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("https://xbib.org/")
.setVersion("HTTP/2.0")
.setResponseListener(resp -> logger.log(Level.INFO, "got HTTP/2 response: " +
resp.getHeaders() + resp.getBodyAsChars(StandardCharsets.UTF_8)))
.build();
client.execute(request).get().close();
}
}
@Test
void testGoogleFollwRedirect() throws Exception {
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("https://google.com")
.setVersion("HTTP/2.0")
.setFollowRedirect(true) // default is true, https://www.google.com/
.setResponseListener(resp -> logger.log(Level.INFO, "got HTTP/2 response: " +
resp.getHeaders() + resp.getBodyAsChars(StandardCharsets.UTF_8)))
.build();
client.execute(request).get().close();
}
}
@Test
void testHttp1WithTlsV13() throws Exception {
AtomicBoolean success = new AtomicBoolean();
NettyHttpClientConfig config = new NettyHttpsClientConfig()
.setSecureProtocolName(new String[] { "TLSv1.3" })
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("https://google.com")
.setVersion("HTTP/2.0")
.setFollowRedirect(true) // default is true, https://www.google.com/
.setResponseListener(resp -> {
logger.log(Level.INFO, "got response: " +
resp.getHeaders() + resp.getBodyAsChars(StandardCharsets.UTF_8));
success.set(true);
})
.build();
client.execute(request).get().close();
}
assertTrue(success.get());
}
@Test
void testParallelRequestsAndClientClose() throws IOException {
AtomicBoolean success1 = new AtomicBoolean();
AtomicBoolean success2 = new AtomicBoolean();
NettyHttpClientConfig config = new NettyHttpsClientConfig();
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request1 = HttpRequest.get()
.setURL("https://google.com")
.setVersion("HTTP/2.0")
.setFollowRedirect(true)
.setResponseListener(resp -> {
logger.log(Level.INFO, "got response1: " +
resp.getHeaders() + resp.getBodyAsChars(StandardCharsets.UTF_8));
success1.set(true);
})
.build();
HttpRequest request2 = HttpRequest.get()
.setURL("https://google.com")
.setVersion("HTTP/2.0")
.setFollowRedirect(true)
.setResponseListener(resp -> {
logger.log(Level.INFO, "got response2: " +
resp.getHeaders() + resp.getBodyAsChars(StandardCharsets.UTF_8));
success2.set(true);
})
.build();
client.execute(request1);
client.execute(request2);
}
assertTrue(success1.get());
assertTrue(success2.get());
}
}

View file

@ -0,0 +1,71 @@
package org.xbib.net.http.netty.client.secure;
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.util.Map;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.junit.jupiter.api.Test;
/**
* Testing the JDK 11+ HTTP client for comparison purposes.
*/
public class JdkClientTest {
private static final Logger logger = Logger.getLogger(JdkClientTest.class.getName());
static {
System.setProperty("javax.net.debug", "true");
}
@Test
public void testDNB() throws Exception {
HttpClient httpClient = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_1_1)
.build();
Map<String, Object> map = Map.of(
"version", "1.1",
"operation", "searchRetrieve",
"recordSchema", "MARC21plus-1-xml",
"query", "iss = 00280836"
);
HttpRequest request = HttpRequest.newBuilder()
.uri(new URI("https://services.dnb.de/sru/zdb"))
.header("Content-Type", "application/x-www-form-urlencoded")
.POST(buildFormDataFromMap(map))
.build();
HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
logger.log(Level.INFO, Integer.toString(response.statusCode()));
logger.log(Level.INFO, response.body());
}
@Test
void testHebisGetRequest() throws Exception {
HttpClient httpClient = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_1_1)
.build();
HttpRequest request = HttpRequest.newBuilder()
.uri(new URI("https://hebis.rz.uni-frankfurt.de/HEBCGI/vuefl_recv_data.pl"))
.build();
HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
logger.log(Level.INFO, Integer.toString(response.statusCode()));
logger.log(Level.INFO, response.body());
}
private static HttpRequest.BodyPublisher buildFormDataFromMap(Map<String, Object> data) {
var builder = new StringBuilder();
for (Map.Entry<String, Object> entry : data.entrySet()) {
if (builder.length() > 0) {
builder.append("&");
}
builder.append(URLEncoder.encode(entry.getKey(), StandardCharsets.UTF_8));
builder.append("=");
builder.append(URLEncoder.encode(entry.getValue().toString(), StandardCharsets.UTF_8));
}
return HttpRequest.BodyPublishers.ofString(builder.toString());
}
}

View file

@ -0,0 +1,76 @@
package org.xbib.net.http.netty.client.secure;
import org.xbib.net.http.client.BackOff;
/**
* Mock for {@link BackOff} that always returns a fixed number.
*
* <p>
* Implementation is not thread-safe.
* </p>
*
*/
public class MockBackOff implements BackOff {
/** Fixed back-off milliseconds. */
private long backOffMillis;
/** Maximum number of tries before returning {@link #STOP}. */
private int maxTries = 10;
/** Number of tries so far. */
private int numTries;
@Override
public void reset() {
numTries = 0;
}
@Override
public long nextBackOffMillis() {
if (numTries >= maxTries || backOffMillis == STOP) {
return STOP;
}
numTries++;
return backOffMillis;
}
/**
* Sets the fixed back-off milliseconds (defaults to {@code 0}).
*
* <p>
* Overriding is only supported for the purpose of calling the super implementation and changing
* the return type, but nothing else.
* </p>
*/
public MockBackOff setBackOffMillis(long backOffMillis) {
//Preconditions.checkArgument(backOffMillis == STOP || backOffMillis >= 0);
this.backOffMillis = backOffMillis;
return this;
}
/**
* Sets the maximum number of tries before returning {@link #STOP} (defaults to {@code 10}).
*
* <p>
* Overriding is only supported for the purpose of calling the super implementation and changing
* the return type, but nothing else.
* </p>
*/
public MockBackOff setMaxTries(int maxTries) {
//Preconditions.checkArgument(maxTries >= 0);
this.maxTries = maxTries;
return this;
}
/** Returns the maximum number of tries before returning {@link #STOP}. */
public final int getMaxTries() {
return numTries;
}
/** Returns the number of tries so far. */
public final int getNumberOfTries() {
return numTries;
}
}

View file

@ -0,0 +1,25 @@
package org.xbib.net.http.netty.client.secure;
import java.io.IOException;
import org.junit.jupiter.api.Test;
import org.xbib.net.http.client.BackOff;
import static org.junit.jupiter.api.Assertions.assertEquals;
/**
* Tests {@link MockBackOff}.
*/
class MockBackOffTest {
@Test
void testNextBackOffMillis() throws IOException {
subtestNextBackOffMillis(0, new MockBackOff());
subtestNextBackOffMillis(BackOff.STOP, new MockBackOff().setBackOffMillis(BackOff.STOP));
subtestNextBackOffMillis(42, new MockBackOff().setBackOffMillis(42));
}
private void subtestNextBackOffMillis(long expectedValue, BackOff backOffPolicy) throws IOException {
for (int i = 0; i < 10; i++) {
assertEquals(expectedValue, backOffPolicy.nextBackOffMillis());
}
}
}

View file

@ -0,0 +1,257 @@
package org.xbib.net.http.netty.client.secure;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.SimpleChannelInboundHandler;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.http.DefaultFullHttpRequest;
import io.netty.handler.codec.http.FullHttpRequest;
import io.netty.handler.codec.http.FullHttpResponse;
import io.netty.handler.codec.http.HttpClientCodec;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http.HttpHeaderValues;
import io.netty.handler.codec.http.HttpMethod;
import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.codec.http.HttpVersion;
import io.netty.util.AttributeKey;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.extension.ExtendWith;
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class SimpleHttp1Test {
private static final Logger logger = Logger.getLogger(SimpleHttp1Test.class.getName());
@AfterAll
void checkThreads() {
Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
logger.log(Level.INFO, "threads = " + threadSet.size() );
threadSet.forEach( thread -> {
if (thread.getName().equals("ObjectCleanerThread")) {
logger.log(Level.INFO, thread.toString());
}
});
}
@Test
void testHttp1() throws Exception {
Client client = new Client();
try {
HttpTransport transport = client.newTransport("google.de", 80);
transport.onResponse(msg -> logger.log(Level.INFO,
"got response: " + msg.status().code() + " headers=" + msg.headers().entries()));
transport.connect();
sendRequest(transport);
transport.awaitResponse();
} finally {
client.shutdown();
}
}
private void sendRequest(HttpTransport transport) {
Channel channel = transport.channel();
if (channel == null) {
return;
}
String host = transport.inetSocketAddress().getHostString();
int port = transport.inetSocketAddress().getPort();
String uri = "http://" + host + ":" + port;
FullHttpRequest request = new DefaultFullHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, uri);
request.headers().add(HttpHeaderNames.HOST, host + ":" + port);
request.headers().add(HttpHeaderNames.USER_AGENT, "Java");
request.headers().add(HttpHeaderNames.ACCEPT_ENCODING, HttpHeaderValues.GZIP);
request.headers().add(HttpHeaderNames.ACCEPT_ENCODING, HttpHeaderValues.DEFLATE);
logger.log(Level.INFO, () -> "writing request = " + request);
if (channel.isWritable()) {
channel.writeAndFlush(request);
}
}
private final AttributeKey<HttpTransport> TRANSPORT_ATTRIBUTE_KEY = AttributeKey.valueOf("transport");
interface ResponseWriter {
void write(FullHttpResponse msg);
}
class Client {
private final EventLoopGroup eventLoopGroup;
private final Bootstrap bootstrap;
private final List<HttpTransport> transports;
Client() {
eventLoopGroup = new NioEventLoopGroup();
HttpResponseHandler httpResponseHandler = new HttpResponseHandler();
Initializer initializer = new Initializer(httpResponseHandler);
bootstrap = new Bootstrap()
.group(eventLoopGroup)
.channel(NioSocketChannel.class)
.handler(initializer);
transports = new ArrayList<>();
}
Bootstrap bootstrap() {
return bootstrap;
}
void shutdown() {
close();
eventLoopGroup.shutdownGracefully();
try {
eventLoopGroup.awaitTermination(10L, TimeUnit.SECONDS);
} catch (InterruptedException e) {
logger.log(Level.WARNING, e.getMessage(), e);
}
}
HttpTransport newTransport(String host, int port) {
HttpTransport transport = new HttpTransport(this, new InetSocketAddress(host, port));
transports.add(transport);
return transport;
}
void close() {
for (HttpTransport transport : transports) {
transport.close();
}
transports.clear();
}
}
class HttpTransport {
private final Client client;
private final InetSocketAddress inetSocketAddress;
private Channel channel;
private CompletableFuture<Boolean> promise;
private ResponseWriter responseWriter;
HttpTransport(Client client, InetSocketAddress inetSocketAddress ) {
this.client = client;
this.inetSocketAddress = inetSocketAddress;
}
InetSocketAddress inetSocketAddress() {
return inetSocketAddress;
}
void connect() throws InterruptedException {
channel = client.bootstrap().connect(inetSocketAddress).sync().await().channel();
channel.attr(TRANSPORT_ATTRIBUTE_KEY).set(this);
promise = new CompletableFuture<>();
}
Channel channel() {
return channel;
}
void onResponse(ResponseWriter responseWriter) {
this.responseWriter = responseWriter;
}
void responseReceived(FullHttpResponse msg) {
if (responseWriter != null) {
responseWriter.write(msg);
}
}
void awaitResponse() {
if (promise != null) {
try {
promise.get(5, TimeUnit.SECONDS);
} catch (InterruptedException | ExecutionException | TimeoutException e) {
logger.log(Level.WARNING, e.getMessage(), e);
}
}
}
void complete() {
if (promise != null) {
promise.complete(true);
}
}
void fail(Throwable throwable) {
if (promise != null) {
promise.completeExceptionally(throwable);
}
}
void close() {
if (channel != null) {
channel.close();
}
}
}
class Initializer extends ChannelInitializer<SocketChannel> {
private HttpResponseHandler httpResponseHandler;
Initializer(HttpResponseHandler httpResponseHandler) {
this.httpResponseHandler = httpResponseHandler;
}
@Override
protected void initChannel(SocketChannel ch) {
ch.pipeline().addLast(new HttpClientCodec());
ch.pipeline().addLast(new HttpObjectAggregator(1048576));
ch.pipeline().addLast(httpResponseHandler);
}
}
class HttpResponseHandler extends SimpleChannelInboundHandler<FullHttpResponse> {
@Override
protected void channelRead0(ChannelHandlerContext ctx, FullHttpResponse msg) {
HttpTransport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
if (msg.content().isReadable()) {
transport.responseReceived(msg);
}
}
@Override
public void channelReadComplete(ChannelHandlerContext ctx) {
HttpTransport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
transport.complete();
}
@Override
public void channelInactive(ChannelHandlerContext ctx) {
ctx.fireChannelInactive();
HttpTransport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
transport.fail(new IOException("channel closed"));
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
logger.log(Level.SEVERE, cause.getMessage(), cause);
HttpTransport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
transport.fail(cause);
ctx.channel().close();
}
}
}

View file

@ -0,0 +1,348 @@
package org.xbib.net.http.netty.client.secure;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.SimpleChannelInboundHandler;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.http.DefaultFullHttpRequest;
import io.netty.handler.codec.http.FullHttpRequest;
import io.netty.handler.codec.http.FullHttpResponse;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http.HttpHeaderValues;
import io.netty.handler.codec.http.HttpMethod;
import io.netty.handler.codec.http.HttpVersion;
import io.netty.handler.codec.http2.DefaultHttp2Connection;
import io.netty.handler.codec.http2.DelegatingDecompressorFrameListener;
import io.netty.handler.codec.http2.Http2ConnectionHandler;
import io.netty.handler.codec.http2.Http2FrameLogger;
import io.netty.handler.codec.http2.Http2SecurityUtil;
import io.netty.handler.codec.http2.Http2Settings;
import io.netty.handler.codec.http2.HttpConversionUtil;
import io.netty.handler.codec.http2.HttpToHttp2ConnectionHandlerBuilder;
import io.netty.handler.codec.http2.InboundHttp2ToHttpAdapterBuilder;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.ssl.ApplicationProtocolConfig;
import io.netty.handler.ssl.ApplicationProtocolNames;
import io.netty.handler.ssl.ApplicationProtocolNegotiationHandler;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.SslProvider;
import io.netty.handler.ssl.SupportedCipherSuiteFilter;
import io.netty.handler.ssl.util.InsecureTrustManagerFactory;
import io.netty.util.AttributeKey;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.SortedMap;
import java.util.TreeMap;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.net.ssl.SSLException;
import org.junit.jupiter.api.Test;
class SimpleHttp2Test {
private static final Logger logger = Logger.getLogger(SimpleHttp2Test.class.getName());
@Test
void testHttp2WithUpgrade() throws Exception {
Client client = new Client();
try {
Http2Transport transport = client.newTransport("webtide.com", 443);
transport.onResponse(string -> logger.log(Level.INFO, "got messsage: " + string));
transport.connect();
transport.awaitSettings();
sendRequest(transport);
transport.awaitResponses();
transport.close();
} finally {
client.shutdown();
}
}
private void sendRequest(Http2Transport transport) {
Channel channel = transport.channel();
if (channel == null) {
return;
}
Integer streamId = transport.nextStream();
String host = transport.inetSocketAddress().getHostString();
int port = transport.inetSocketAddress().getPort();
String uri = "https://" + host + ":" + port;
FullHttpRequest request = new DefaultFullHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, uri);
request.headers().add(HttpHeaderNames.HOST, host + ":" + port);
request.headers().add(HttpHeaderNames.USER_AGENT, "Java");
request.headers().add(HttpHeaderNames.ACCEPT_ENCODING, HttpHeaderValues.GZIP);
request.headers().add(HttpHeaderNames.ACCEPT_ENCODING, HttpHeaderValues.DEFLATE);
if (streamId != null) {
request.headers().add(HttpConversionUtil.ExtensionHeaderNames.STREAM_ID.text(), Integer.toString(streamId));
}
logger.log(Level.INFO, () -> "writing request = " + request);
channel.writeAndFlush(request);
}
private final AttributeKey<Http2Transport> TRANSPORT_ATTRIBUTE_KEY = AttributeKey.valueOf("transport");
interface ResponseWriter {
void write(String string);
}
class Client {
private final EventLoopGroup eventLoopGroup;
private final Bootstrap bootstrap;
Client() {
eventLoopGroup = new NioEventLoopGroup();
Http2SettingsHandler http2SettingsHandler = new Http2SettingsHandler();
Http2ResponseHandler http2ResponseHandler = new Http2ResponseHandler();
Initializer initializer = new Initializer(http2SettingsHandler, http2ResponseHandler);
bootstrap = new Bootstrap()
.group(eventLoopGroup)
.channel(NioSocketChannel.class)
.handler(initializer);
}
Bootstrap bootstrap() {
return bootstrap;
}
void shutdown() {
eventLoopGroup.shutdownGracefully();
}
Http2Transport newTransport(String host, int port) {
return new Http2Transport(this, new InetSocketAddress(host, port));
}
}
class Http2Transport {
private final Client client;
private final InetSocketAddress inetSocketAddress;
private Channel channel;
CompletableFuture<Boolean> settingsPromise;
private final SortedMap<Integer, CompletableFuture<Boolean>> streamidPromiseMap;
private final AtomicInteger streamIdCounter;
private ResponseWriter responseWriter;
Http2Transport(Client client, InetSocketAddress inetSocketAddress) {
this.client = client;
this.inetSocketAddress = inetSocketAddress;
streamidPromiseMap = new TreeMap<>();
streamIdCounter = new AtomicInteger(3);
}
InetSocketAddress inetSocketAddress() {
return inetSocketAddress;
}
void connect() throws InterruptedException {
channel = client.bootstrap().connect(inetSocketAddress).sync().await().channel();
channel.attr(TRANSPORT_ATTRIBUTE_KEY).set(this);
settingsPromise = new CompletableFuture<>();
}
Channel channel() {
return channel;
}
Integer nextStream() {
Integer streamId = streamIdCounter.getAndAdd(2);
streamidPromiseMap.put(streamId, new CompletableFuture<>());
return streamId;
}
void onResponse(ResponseWriter responseWriter) {
this.responseWriter = responseWriter;
}
void settingsReceived(Channel channel, Http2Settings http2Settings) {
if (settingsPromise != null) {
settingsPromise.complete(true);
} else {
logger.log(Level.WARNING, "settings received but no promise present");
}
}
void awaitSettings() {
if (settingsPromise != null) {
try {
logger.log(Level.INFO, "waiting for settings");
settingsPromise.get(5, TimeUnit.SECONDS);
logger.log(Level.INFO, "settings received");
} catch (InterruptedException | ExecutionException | TimeoutException e) {
settingsPromise.completeExceptionally(e);
}
} else {
logger.log(Level.WARNING, "waiting for settings but no promise present");
}
}
void responseReceived(Integer streamId, String message) {
if (streamId == null) {
logger.log(Level.WARNING, "unexpected message received: " + message);
return;
}
CompletableFuture<Boolean> promise = streamidPromiseMap.get(streamId);
if (promise == null) {
logger.log(Level.WARNING, "message received for unknown stream id " + streamId);
} else {
if (responseWriter != null) {
responseWriter.write(message);
}
promise.complete(true);
}
}
void awaitResponse(Integer streamId) {
if (streamId == null) {
return;
}
CompletableFuture<Boolean> promise = streamidPromiseMap.get(streamId);
if (promise != null) {
try {
logger.log(Level.INFO, "waiting for response for stream id=" + streamId);
promise.get(5, TimeUnit.SECONDS);
logger.log(Level.INFO, "response for stream id=" + streamId + " received");
} catch (InterruptedException | ExecutionException | TimeoutException e) {
logger.log(Level.WARNING, "streamId=" + streamId + " " + e.getMessage(), e);
} finally {
streamidPromiseMap.remove(streamId);
}
}
}
void awaitResponses() {
logger.log(Level.INFO, "waiting for all stream ids " + streamidPromiseMap.keySet());
for (int streamId : streamidPromiseMap.keySet()) {
awaitResponse(streamId);
}
}
void fail(Throwable throwable) {
for (CompletableFuture<Boolean> promise : streamidPromiseMap.values()) {
promise.completeExceptionally(throwable);
}
}
void close() {
if (channel != null) {
channel.close();
}
}
}
class Initializer extends ChannelInitializer<SocketChannel> {
private final Http2SettingsHandler http2SettingsHandler;
private final Http2ResponseHandler http2ResponseHandler;
Initializer(Http2SettingsHandler http2SettingsHandler, Http2ResponseHandler http2ResponseHandler) {
this.http2SettingsHandler = http2SettingsHandler;
this.http2ResponseHandler = http2ResponseHandler;
}
@Override
protected void initChannel(SocketChannel ch) {
DefaultHttp2Connection http2Connection = new DefaultHttp2Connection(false);
Http2FrameLogger frameLogger = new Http2FrameLogger(LogLevel.INFO, "client");
Http2ConnectionHandler http2ConnectionHandler = new HttpToHttp2ConnectionHandlerBuilder()
.connection(http2Connection)
.frameLogger(frameLogger)
.frameListener(new DelegatingDecompressorFrameListener(http2Connection,
new InboundHttp2ToHttpAdapterBuilder(http2Connection)
.maxContentLength(10 * 1024 * 1024)
.propagateSettings(true)
.build()))
.build();
try {
SslContext sslContext = SslContextBuilder.forClient()
.sslProvider(SslProvider.JDK)
.trustManager(InsecureTrustManagerFactory.INSTANCE)
.ciphers(Http2SecurityUtil.CIPHERS, SupportedCipherSuiteFilter.INSTANCE)
.applicationProtocolConfig(new ApplicationProtocolConfig(
ApplicationProtocolConfig.Protocol.ALPN,
ApplicationProtocolConfig.SelectorFailureBehavior.NO_ADVERTISE,
ApplicationProtocolConfig.SelectedListenerFailureBehavior.ACCEPT,
ApplicationProtocolNames.HTTP_2))
.build();
ch.pipeline().addLast(sslContext.newHandler(ch.alloc()));
ApplicationProtocolNegotiationHandler negotiationHandler = new ApplicationProtocolNegotiationHandler("") {
@Override
protected void configurePipeline(ChannelHandlerContext ctx, String protocol) {
if (ApplicationProtocolNames.HTTP_2.equals(protocol)) {
ctx.pipeline().addLast(http2ConnectionHandler, http2SettingsHandler, http2ResponseHandler);
return;
}
ctx.close();
throw new IllegalStateException("unknown protocol: " + protocol);
}
};
ch.pipeline().addLast(negotiationHandler);
} catch (SSLException e) {
logger.log(Level.SEVERE, e.getMessage(), e);
}
}
}
class Http2SettingsHandler extends SimpleChannelInboundHandler<Http2Settings> {
@Override
protected void channelRead0(ChannelHandlerContext ctx, Http2Settings http2Settings) {
Http2Transport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
transport.settingsReceived(ctx.channel(), http2Settings);
ctx.pipeline().remove(this);
}
}
class Http2ResponseHandler extends SimpleChannelInboundHandler<FullHttpResponse> {
@Override
protected void channelRead0(ChannelHandlerContext ctx, FullHttpResponse msg) {
Http2Transport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
Integer streamId = msg.headers().getInt(HttpConversionUtil.ExtensionHeaderNames.STREAM_ID.text());
if (msg.content().isReadable()) {
transport.responseReceived(streamId, msg.content().toString(StandardCharsets.UTF_8));
}
}
@Override
public void channelReadComplete(ChannelHandlerContext ctx) {
// do nothing
}
@Override
public void channelInactive(ChannelHandlerContext ctx) {
ctx.fireChannelInactive();
Http2Transport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
transport.fail(new IOException("channel closed"));
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
logger.log(Level.SEVERE, cause.getMessage(), cause);
Http2Transport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
transport.fail(cause);
ctx.channel().close();
}
}
}

View file

@ -0,0 +1,36 @@
package org.xbib.net.http.netty.client.secure;
import java.io.IOException;
import java.util.Set;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.secure.NettyHttpsClientConfig;
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
class ThreadLeakTest {
private static final Logger logger = Logger.getLogger(ThreadLeakTest.class.getName());
@Test
void testForLeaks() throws IOException {
NettyHttpClientConfig config = new NettyHttpsClientConfig();
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
}
}
@BeforeAll
@AfterAll
void checkThreads() {
Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
logger.log(Level.INFO, "threads = " + threadSet.size() );
threadSet.forEach( thread -> logger.log(Level.INFO, thread.toString()));
}
}

View file

@ -0,0 +1,369 @@
package org.xbib.net.http.netty.client.secure;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.SimpleChannelInboundHandler;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.codec.http.DefaultFullHttpRequest;
import io.netty.handler.codec.http.FullHttpRequest;
import io.netty.handler.codec.http.FullHttpResponse;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http.HttpMethod;
import io.netty.handler.codec.http.HttpVersion;
import io.netty.handler.codec.http2.DefaultHttp2Connection;
import io.netty.handler.codec.http2.DefaultHttp2SettingsFrame;
import io.netty.handler.codec.http2.DelegatingDecompressorFrameListener;
import io.netty.handler.codec.http2.Http2ConnectionHandler;
import io.netty.handler.codec.http2.Http2ConnectionPrefaceAndSettingsFrameWrittenEvent;
import io.netty.handler.codec.http2.Http2FrameLogger;
import io.netty.handler.codec.http2.Http2SecurityUtil;
import io.netty.handler.codec.http2.Http2Settings;
import io.netty.handler.codec.http2.HttpConversionUtil;
import io.netty.handler.codec.http2.HttpToHttp2ConnectionHandlerBuilder;
import io.netty.handler.codec.http2.InboundHttp2ToHttpAdapterBuilder;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.ssl.ApplicationProtocolConfig;
import io.netty.handler.ssl.ApplicationProtocolNames;
import io.netty.handler.ssl.ApplicationProtocolNegotiationHandler;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.SslProvider;
import io.netty.handler.ssl.SupportedCipherSuiteFilter;
import io.netty.handler.ssl.util.InsecureTrustManagerFactory;
import io.netty.util.AttributeKey;
import java.io.Closeable;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets;
import java.util.SortedMap;
import java.util.TreeMap;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.net.ssl.SSLException;
import org.junit.jupiter.api.Test;
class WebtideTest {
private static final Logger logger = Logger.getLogger(WebtideTest.class.getName());
/**
* Netty standalone demo to connect to <a href="https://webtide.com">https://webtide.com</a>
* and negotiate HTTP/2 and receive responses as HTTP objects.
*/
@Test
void testWebtideHttps() throws Exception {
try (Client client = new Client()) {
InetSocketAddress address = new InetSocketAddress("google.com", 443);
Http2Transport transport = new Http2Transport(client.bootstrap, address);
transport.onResponse(string -> logger.log(Level.INFO, "got response for request = " + string));
logger.log(Level.FINE, "connected");
transport.connect();
logger.log(Level.FINE, "waiting for settings");
transport.awaitSettings();
sendRequest(transport);
logger.log(Level.FINE, "waiting for responses");
transport.awaitResponses();
logger.log(Level.FINE, "close");
transport.close();
}
}
private void sendRequest(Http2Transport transport) {
Channel channel = transport.channel();
if (channel == null) {
return;
}
Integer streamId = transport.nextStream();
String host = transport.inetSocketAddress().getHostString();
int port = transport.inetSocketAddress().getPort();
String uri = "https://" + host + ":" + port;
FullHttpRequest request = new DefaultFullHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, uri);
request.headers().add(HttpHeaderNames.HOST, host + ":" + port);
request.headers().add(HttpHeaderNames.USER_AGENT, "Java");
if (streamId != null) {
request.headers().add(HttpConversionUtil.ExtensionHeaderNames.STREAM_ID.text(), Integer.toString(streamId));
}
logger.log(Level.FINE, "request prepared and ready for sending " + request);
channel.writeAndFlush(request);
}
private final AttributeKey<Http2Transport> TRANSPORT_ATTRIBUTE_KEY = AttributeKey.valueOf("transport");
interface ResponseWriter {
void write(String string);
}
class Client implements Closeable {
private final EventLoopGroup eventLoopGroup;
private final Bootstrap bootstrap;
Client() {
eventLoopGroup = new NioEventLoopGroup();
Initializer initializer = new Initializer();
bootstrap = new Bootstrap()
.group(eventLoopGroup)
.channel(NioSocketChannel.class)
.handler(initializer);
}
@Override
public void close() throws IOException {
eventLoopGroup.shutdownGracefully();
}
}
class Http2Transport {
private final Bootstrap bootstrap;
private final InetSocketAddress inetSocketAddress;
private final SortedMap<Integer, CompletableFuture<Boolean>> streamidPromiseMap;
private final AtomicInteger streamIdCounter;
private final CompletableFuture<Boolean> settingsPromise;
private Channel channel;
private ResponseWriter responseWriter;
Http2Transport(Bootstrap bootstrap, InetSocketAddress inetSocketAddress) {
this.bootstrap = bootstrap;
this.inetSocketAddress = inetSocketAddress;
this.streamidPromiseMap = new TreeMap<>();
this.streamIdCounter = new AtomicInteger(3);
this.settingsPromise = new CompletableFuture<>();
}
InetSocketAddress inetSocketAddress() {
return inetSocketAddress;
}
void connect() throws InterruptedException {
channel = bootstrap.connect(inetSocketAddress).sync().await().channel();
channel.attr(TRANSPORT_ATTRIBUTE_KEY).set(this);
}
Channel channel() {
return channel;
}
Integer nextStream() {
Integer streamId = streamIdCounter.getAndAdd(2);
streamidPromiseMap.put(streamId, new CompletableFuture<>());
return streamId;
}
void onResponse(ResponseWriter responseWriter) {
this.responseWriter = responseWriter;
}
void settingsReceived(Channel channel, Http2Settings http2Settings) {
settingsPromise.complete(true);
}
void awaitSettings() {
try {
settingsPromise.get(5, TimeUnit.SECONDS);
logger.log(Level.INFO, "settings received");
} catch (InterruptedException | ExecutionException | TimeoutException e) {
settingsPromise.completeExceptionally(e);
}
}
void responseReceived(Integer streamId, String message) {
if (streamId == null) {
logger.log(Level.WARNING, "unexpected message received: " + message);
return;
}
CompletableFuture<Boolean> promise = streamidPromiseMap.get(streamId);
if (promise == null) {
logger.log(Level.WARNING, "message received for unknown stream id " + streamId);
} else {
if (responseWriter != null) {
responseWriter.write(message);
}
promise.complete(true);
}
}
void awaitResponse(Integer streamId) {
if (streamId == null) {
return;
}
CompletableFuture<Boolean> promise = streamidPromiseMap.get(streamId);
if (promise != null) {
try {
logger.log(Level.INFO, "waiting for response for stream id=" + streamId);
promise.get(5, TimeUnit.SECONDS);
logger.log(Level.INFO, "response for stream id=" + streamId + " received");
} catch (InterruptedException | ExecutionException | TimeoutException e) {
logger.log(Level.WARNING, "streamId=" + streamId + " " + e.getMessage(), e);
} finally {
streamidPromiseMap.remove(streamId);
}
}
}
void awaitResponses() {
logger.log(Level.INFO, "waiting for all stream ids " + streamidPromiseMap.keySet());
for (int streamId : streamidPromiseMap.keySet()) {
awaitResponse(streamId);
}
}
void fail(Throwable throwable) {
for (CompletableFuture<Boolean> promise : streamidPromiseMap.values()) {
promise.completeExceptionally(throwable);
}
}
void close() {
if (channel != null) {
channel.close();
}
}
}
class Initializer extends ChannelInitializer<SocketChannel> {
@Override
protected void initChannel(SocketChannel ch) {
try {
SslContext sslContext = SslContextBuilder.forClient()
.sslProvider(SslProvider.JDK)
.trustManager(InsecureTrustManagerFactory.INSTANCE)
.ciphers(Http2SecurityUtil.CIPHERS, SupportedCipherSuiteFilter.INSTANCE)
.applicationProtocolConfig(new ApplicationProtocolConfig(
ApplicationProtocolConfig.Protocol.ALPN,
ApplicationProtocolConfig.SelectorFailureBehavior.NO_ADVERTISE,
ApplicationProtocolConfig.SelectedListenerFailureBehavior.ACCEPT,
ApplicationProtocolNames.HTTP_2))
.build();
ch.pipeline().addLast(sslContext.newHandler(ch.alloc()));
ApplicationProtocolNegotiationHandler negotiationHandler = new ApplicationProtocolNegotiationHandler("") {
@Override
protected void configurePipeline(ChannelHandlerContext ctx, String protocol) {
logger.log(Level.INFO, "ALPN negotiated protocol = " + protocol);
if (ApplicationProtocolNames.HTTP_2.equals(protocol)) {
DefaultHttp2Connection http2Connection = new DefaultHttp2Connection(false);
Http2FrameLogger frameLogger = new Http2FrameLogger(LogLevel.INFO, "client");
Http2ConnectionHandler http2ConnectionHandler = new HttpToHttp2ConnectionHandlerBuilder()
.connection(http2Connection)
.frameLogger(frameLogger)
.frameListener(new DelegatingDecompressorFrameListener(http2Connection,
new InboundHttp2ToHttpAdapterBuilder(http2Connection)
.maxContentLength(10 * 1024 * 1024)
.propagateSettings(true)
.build()))
.build();
Http2SettingsHandler http2SettingsHandler = new Http2SettingsHandler();
Http2ResponseHandler http2ResponseHandler = new Http2ResponseHandler();
Http2ResponseMessages http2ResponseMessages = new Http2ResponseMessages();
ctx.pipeline().addLast(http2ConnectionHandler, http2SettingsHandler,
http2ResponseHandler, http2ResponseMessages);
logger.log(Level.INFO, "HTTP/2 pipeline set up = " + ctx.channel().pipeline().names());
return;
}
ctx.close();
throw new IllegalStateException("unknown protocol: " + protocol);
}
};
ch.pipeline().addLast(negotiationHandler);
} catch (SSLException e) {
logger.log(Level.SEVERE, e.getMessage(), e);
}
}
}
class Http2SettingsHandler extends SimpleChannelInboundHandler<Http2Settings> {
@Override
protected void channelRead0(ChannelHandlerContext ctx, Http2Settings http2Settings) {
logger.log(Level.INFO, "got settings = " + http2Settings);
Http2Transport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
transport.settingsReceived(ctx.channel(), http2Settings);
ctx.pipeline().remove(this);
}
}
class Http2ResponseHandler extends SimpleChannelInboundHandler<FullHttpResponse> {
@Override
protected void channelRead0(ChannelHandlerContext ctx, FullHttpResponse msg) {
logger.log(Level.INFO, "got full http response = " + msg);
Http2Transport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
Integer streamId = msg.headers().getInt(HttpConversionUtil.ExtensionHeaderNames.STREAM_ID.text());
if (msg.content().isReadable()) {
transport.responseReceived(streamId, msg.content().toString(StandardCharsets.UTF_8));
}
}
@Override
public void channelReadComplete(ChannelHandlerContext ctx) {
logger.log(Level.INFO, "channel read complete");
// do nothing
}
@Override
public void channelInactive(ChannelHandlerContext ctx) {
logger.log(Level.INFO, "channel inactive");
ctx.fireChannelInactive();
Http2Transport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
transport.fail(new IOException("channel closed"));
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
logger.log(Level.SEVERE, cause.getMessage(), cause);
Http2Transport transport = ctx.channel().attr(TRANSPORT_ATTRIBUTE_KEY).get();
transport.fail(cause);
ctx.channel().close();
}
}
class Http2ResponseMessages extends ChannelInboundHandlerAdapter {
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
logger.log(Level.FINEST, "received msg = " + msg.getClass().getName());
if (msg instanceof DefaultHttp2SettingsFrame) {
DefaultHttp2SettingsFrame settingsFrame = (DefaultHttp2SettingsFrame) msg;
logger.log(Level.FINEST, "received settings ");
}
}
@Override
public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception {
logger.log(Level.FINEST, "received event = " + evt.getClass().getName());
if (evt instanceof Http2ConnectionPrefaceAndSettingsFrameWrittenEvent) {
Http2ConnectionPrefaceAndSettingsFrameWrittenEvent event =
(Http2ConnectionPrefaceAndSettingsFrameWrittenEvent) evt;
logger.log(Level.FINEST, "received preface and setting written event " + event);
}
ctx.fireUserEventTriggered(evt);
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
logger.log(Level.FINEST, "received exception " + cause);
}
}
}

View file

@ -0,0 +1 @@
org.xbib.net.http.netty.boringssl.BoringSSLClientSecureSocketProvider

View file

@ -0,0 +1,5 @@
handlers=java.util.logging.ConsoleHandler
.level=ALL
java.util.logging.ConsoleHandler.level=ALL
java.util.logging.ConsoleHandler.formatter=org.xbib.net.util.ThreadLoggingFormatter
jdk.event.security.level=INFO

View file

@ -0,0 +1,5 @@
dependencies {
api project(':net-http-client')
api libs.netty.codec.http2
api libs.netty.handler.proxy
}

View file

@ -0,0 +1,27 @@
import org.xbib.net.http.client.netty.ClientTransportProvider;
import org.xbib.net.http.client.netty.http1.Http1ChannelInitializer;
import org.xbib.net.http.client.netty.http2.Http2ChannelInitializer;
import org.xbib.net.http.client.netty.HttpChannelInitializer;
import org.xbib.net.http.client.netty.NioClientTransportProvider;
module org.xbib.net.http.client.netty {
exports org.xbib.net.http.client.netty;
exports org.xbib.net.http.client.netty.http1;
exports org.xbib.net.http.client.netty.http2;
requires org.xbib.net;
requires org.xbib.net.http;
requires org.xbib.net.http.client;
requires io.netty.buffer;
requires io.netty.common;
requires io.netty.transport;
requires io.netty.handler;
requires io.netty.codec;
requires io.netty.codec.http;
requires io.netty.codec.http2;
requires io.netty.handler.proxy;
requires java.logging;
uses ClientTransportProvider;
provides ClientTransportProvider with NioClientTransportProvider;
uses HttpChannelInitializer;
provides HttpChannelInitializer with Http1ChannelInitializer, Http2ChannelInitializer;
}

View file

@ -0,0 +1,429 @@
package org.xbib.net.http.client.netty;
import io.netty.channel.Channel;
import io.netty.channel.ChannelPromise;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http2.Http2Settings;
import java.io.IOException;
import java.nio.charset.MalformedInputException;
import java.nio.charset.StandardCharsets;
import java.nio.charset.UnmappableCharacterException;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.function.Function;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.stream.Collectors;
import org.xbib.net.PercentDecoder;
import org.xbib.net.URL;
import org.xbib.net.URLSyntaxException;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpMethod;
import org.xbib.net.http.client.BackOff;
import org.xbib.net.http.client.HttpResponse;
import org.xbib.net.http.cookie.Cookie;
import org.xbib.net.http.cookie.CookieBox;
public abstract class BaseInteraction implements Interaction {
private static final Logger logger = Logger.getLogger(BaseInteraction.class.getName());
protected final NettyHttpClient nettyHttpClient;
protected final HttpAddress httpAddress;
protected Throwable throwable;
protected final Map<String, StreamIds> streamIds;
protected HttpRequest httpRequest;
protected Channel channel;
private CookieBox cookieBox;
protected ChannelPromise settingsPromise;
protected Http2Settings http2Settings;
protected CompletableFuture<?> future;
public BaseInteraction(NettyHttpClient nettyHttpClient, HttpAddress httpAddress) {
this.nettyHttpClient = nettyHttpClient;
this.httpAddress = httpAddress;
this.streamIds = new ConcurrentHashMap<>();
}
@Override
public void setSettingsPromise(ChannelPromise settingsPromise) {
this.settingsPromise = settingsPromise;
}
@Override
public HttpAddress getHttpAddress() {
return httpAddress;
}
public void setFuture(CompletableFuture<?> future) {
this.future = future;
}
public CompletableFuture<?> getFuture() {
return future;
}
/**
* Method for executing the request and respond in a completable future.
*
* @param request request
* @param supplier supplier
* @param <T> supplier result
* @return completable future
*/
@Override
public <T> CompletableFuture<T> execute(HttpRequest request, Function<HttpResponse, T> supplier)
throws IOException {
Objects.requireNonNull(request);
this.httpRequest = request;
Objects.requireNonNull(supplier);
final CompletableFuture<T> completableFuture = new CompletableFuture<>();
request.setResponseListener(response -> {
if (response != null) {
completableFuture.complete(supplier.apply(response));
} else {
completableFuture.cancel(true);
}
get();
cancel();
});
request.setTimeoutListener(req -> completableFuture.completeExceptionally(new TimeoutException()));
request.setExceptionListener(completableFuture::completeExceptionally);
execute(request);
return completableFuture;
}
@Override
public void close() throws IOException {
logger.log(Level.FINE, "closing interaction " + this);
get();
//cancel();
releaseChannel(channel, true);
if (future != null) {
future.complete(null);
}
}
@Override
public boolean isFailed() {
return throwable != null;
}
@Override
public Throwable getFailure() {
return throwable;
}
/**
* The underlying network layer failed.
* So we fail all (open) promises.
* @param throwable the exception
*/
@Override
public void fail(Channel channel, Throwable throwable) {
// do not fail more than once
if (this.throwable != null) {
return;
}
this.throwable = throwable;
logger.log(Level.SEVERE, "channel " + channel + " failing: " + throwable.getMessage(), throwable);
for (StreamIds streamIds : streamIds.values()) {
streamIds.fail(throwable);
}
if (future != null) {
future.completeExceptionally(throwable);
}
}
@Override
public void inactive(Channel channel) {
// do nothing
}
@Override
public Interaction get() {
return get(nettyHttpClient.getClientConfig().getSocketConfig().getReadTimeoutMillis(), TimeUnit.MILLISECONDS);
}
@Override
public Interaction get(long value, TimeUnit timeUnit) {
if (!streamIds.isEmpty()) {
for (Map.Entry<String, StreamIds> entry : streamIds.entrySet()) {
StreamIds streamIds = entry.getValue();
if (!streamIds.isClosed()) {
for (Integer key : streamIds.keys()) {
String requestKey = getRequestKey(entry.getKey(), key);
try {
CompletableFuture<Boolean> timeoutFuture = streamIds.get(key);
Boolean timeout = timeoutFuture.get(value, timeUnit);
if (timeout) {
completeRequest(requestKey);
} else {
completeRequestTimeout(requestKey, new TimeoutException());
}
} catch (TimeoutException e) {
completeRequestTimeout(requestKey, new TimeoutException());
} catch (Exception e) {
completeRequestExceptionally(requestKey, e);
streamIds.fail(e);
} finally {
streamIds.remove(key);
}
}
streamIds.close();
}
}
}
nettyHttpClient.remove(this);
return this;
}
@Override
public void cancel() {
if (!streamIds.isEmpty()) {
for (Map.Entry<String, StreamIds> entry : streamIds.entrySet()) {
StreamIds streamIds = entry.getValue();
for (Integer key : streamIds.keys()) {
try {
streamIds.get(key).cancel(true);
} catch (Exception e) {
completeRequestExceptionally(getRequestKey(entry.getKey(), key), e);
streamIds.fail(e);
} finally {
streamIds.remove(key);
}
}
streamIds.close();
}
streamIds.clear();
}
}
protected abstract String getRequestKey(String channelId, Integer streamId);
protected Channel acquireChannel(HttpRequest request) throws IOException {
Channel channel;
if (nettyHttpClient.hasPooledNodes()) {
channel = nextChannel();
this.channel = channel;
} else {
channel = this.channel;
if (channel == null) {
channel = nextChannel();
}
this.channel = channel;
}
return channel;
}
protected Channel newChannel(HttpAddress httpAddress) throws IOException {
if (httpAddress != null) {
try {
return nettyHttpClient.getBootstrap()
.handler(nettyHttpClient.newChannelInitializer(httpAddress, this))
.connect(httpAddress.getInetSocketAddress()).sync().await().channel();
} catch (InterruptedException e) {
throw new IOException(e);
}
} else {
if (nettyHttpClient.hasPooledNodes()) {
try {
return nettyHttpClient.getPool().acquire();
} catch (Exception e) {
throw new IOException(e);
}
} else {
throw new UnsupportedOperationException();
}
}
}
protected void releaseChannel(Channel channel, boolean close) throws IOException{
if (channel == null) {
return;
}
if (nettyHttpClient.hasPooledNodes()) {
try {
nettyHttpClient.getPool().release(channel, close);
} catch (Exception e) {
throw new IOException(e);
}
} else if (close) {
channel.close();
}
}
protected abstract Channel nextChannel() throws IOException;
protected HttpRequest continuation(HttpRequest request, HttpResponse httpResponse) throws URLSyntaxException {
if (httpResponse == null) {
return null;
}
if (request == null) {
// push promise or something else
return null;
}
try {
if (request.canRedirect()) {
int status = httpResponse.getStatus().code();
switch (status) {
case 300:
case 301:
case 302:
case 303:
case 305:
case 307:
case 308:
String location = httpResponse.getHeaders().get(HttpHeaderNames.LOCATION);
location = new PercentDecoder(StandardCharsets.UTF_8.newDecoder()).decode(location);
if (location != null) {
logger.log(Level.FINE, "found redirect location: " + location);
URL redirUrl = URL.base(request.getURL()).resolve(location);
HttpMethod method = httpResponse.getStatus().code() == 303 ? HttpMethod.GET : request.getMethod();
HttpRequestBuilder newHttpRequestHttpRequestBuilder = HttpRequest.builder(method, request)
.setURL(redirUrl);
request.getURL().getQueryParams().forEach(pair ->
newHttpRequestHttpRequestBuilder.addParameter(pair.getKey(), pair.getValue())
);
request.cookies().forEach(newHttpRequestHttpRequestBuilder::addCookie);
HttpRequest newHttpRequest = newHttpRequestHttpRequestBuilder.build();
StringBuilder hostAndPort = new StringBuilder();
hostAndPort.append(redirUrl.getHost());
if (redirUrl.getPort() != null) {
hostAndPort.append(':').append(redirUrl.getPort());
}
newHttpRequest.getHeaders().set(HttpHeaderNames.HOST, hostAndPort.toString());
logger.log(Level.FINE, "redirect url: " + redirUrl);
return newHttpRequest;
}
break;
default:
break;
}
}
} catch (MalformedInputException | UnmappableCharacterException e) {
this.throwable = e;
}
return null;
}
protected HttpRequest retry(HttpRequest request, HttpResponse httpResponse) {
if (httpResponse == null) {
// no response present, invalid in any way
return null;
}
if (request == null) {
// push promise or something else
return null;
}
if (request.isBackOff()) {
BackOff backOff = request.getBackOff() != null ?
request.getBackOff() :
nettyHttpClient.getClientConfig().getBackOff();
int status = httpResponse.getStatus ().code();
switch (status) {
case 403:
case 404:
case 500:
case 502:
case 503:
case 504:
case 507:
case 509:
if (backOff != null) {
long millis = backOff.nextBackOffMillis();
if (millis != BackOff.STOP) {
logger.log(Level.FINE, () -> "status = " + status + " backing off request by " + millis + " milliseconds");
try {
Thread.sleep(millis);
} catch (InterruptedException e) {
// ignore
}
return request;
}
}
break;
default:
break;
}
}
return null;
}
private void completeRequest(String requestKey) {
if (requestKey != null) {
if (httpRequest != null && httpRequest.getCompletableFuture() != null) {
httpRequest.getCompletableFuture().complete(httpRequest);
}
}
}
private void completeRequestExceptionally(String requestKey, Throwable throwable) {
if (requestKey != null) {
httpRequest.onException(throwable);
}
}
private void completeRequestTimeout(String requestKey, TimeoutException timeoutException) {
if (requestKey != null) {
httpRequest.onTimeout();
}
}
@Override
public void setCookieBox(CookieBox cookieBox) {
this.cookieBox = cookieBox;
}
@Override
public CookieBox getCookieBox() {
return cookieBox;
}
protected void addCookie(Cookie cookie) {
if (cookieBox == null) {
this.cookieBox = new CookieBox();
}
cookieBox.add(cookie);
}
protected List<Cookie> matchCookiesFromBox(HttpRequest request) {
return cookieBox == null ? Collections.emptyList() : cookieBox.stream().filter(cookie ->
matchCookie(request.getURL(), cookie)).collect(Collectors.toList());
}
protected List<Cookie> matchCookies(HttpRequest request) {
return request.cookies().stream().filter(cookie ->
matchCookie(request.getURL(), cookie)).collect(Collectors.toList());
}
private boolean matchCookie(URL url, Cookie cookie) {
boolean domainMatch = cookie.domain() == null || url.getHost().endsWith(cookie.domain());
if (!domainMatch) {
return false;
}
if (cookie.path() != null) {
boolean pathMatch = "/".equals(cookie.path()) || url.getPath().startsWith(cookie.path());
if (!pathMatch) {
return false;
}
}
boolean secureScheme = "https".equals(url.getScheme());
return (secureScheme && cookie.isSecure()) || (!secureScheme && !cookie.isSecure());
}
}

View file

@ -0,0 +1,395 @@
package org.xbib.net.http.client.netty;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelFutureListener;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPromise;
import io.netty.channel.pool.ChannelPoolHandler;
import io.netty.channel.socket.SocketChannel;
import io.netty.handler.codec.http2.DefaultHttp2GoAwayFrame;
import java.io.IOException;
import java.net.ConnectException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Queue;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Semaphore;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpVersion;
public class BoundedChannelPool implements Pool {
private static final Logger logger = Logger.getLogger(BoundedChannelPool.class.getName());
private final Semaphore semaphore;
private final HttpVersion httpVersion;
private ChannelPoolHandler channelPoolhandler;
private final List<HttpAddress> nodes;
private final int numberOfNodes;
private final int retriesPerNode;
private final Map<HttpAddress, Bootstrap> bootstraps;
private final Map<HttpAddress, List<Channel>> channels;
private final Map<HttpAddress, Queue<Channel>> availableChannels;
private final Map<HttpAddress, Integer> counts;
private final Map<HttpAddress, Integer> failedCounts;
private final Lock lock;
private PoolKeySelector poolKeySelector;
/**
* A bounded channel pool.
*
* @param semaphore the level of concurrency
* @param httpVersion the HTTP version of the pool connections
* @param nodes the endpoint nodes, any element may contain the port (followed after ":")
* to override the defaultPort argument
* @param retriesPerNode the max count of the subsequent connection failures to the node before
* the node will be excluded from the pool. If set to 0, the value is ignored.
* @param poolKeySelectorType pool key selector type
*/
public BoundedChannelPool(Semaphore semaphore,
HttpVersion httpVersion,
List<HttpAddress> nodes,
int retriesPerNode,
PoolKeySelectorType poolKeySelectorType) {
this.semaphore = semaphore;
this.httpVersion = httpVersion;
this.nodes = nodes;
this.retriesPerNode = retriesPerNode;
switch (poolKeySelectorType) {
case RANDOM:
this.poolKeySelector = new RandomPoolKeySelector();
break;
case ROUNDROBIN:
this.poolKeySelector = new RoundRobinKeySelector();
break;
}
this.lock = new ReentrantLock();
if (nodes == null || nodes.isEmpty()) {
throw new IllegalArgumentException("nodes must not be empty");
}
this.numberOfNodes = nodes.size();
bootstraps = new HashMap<>(numberOfNodes);
channels = new ConcurrentHashMap<>(numberOfNodes);
availableChannels = new ConcurrentHashMap<>(numberOfNodes);
counts = new ConcurrentHashMap<>(numberOfNodes);
failedCounts = new ConcurrentHashMap<>(numberOfNodes);
}
/**
* Initialize pool.
*
* @param bootstrap bootstrap instance
* @param channelPoolHandler channel pool handler being notified upon new connection is created
*/
public void init(Bootstrap bootstrap, ChannelPoolHandler channelPoolHandler, int channelCount) throws IOException {
this.channelPoolhandler = channelPoolHandler;
for (HttpAddress node : nodes) {
HttpChannelPoolInitializer initializer = new HttpChannelPoolInitializer(node, channelPoolHandler);
bootstraps.put(node, bootstrap.clone().remoteAddress(node.getInetSocketAddress())
.handler(initializer));
availableChannels.put(node, new ConcurrentLinkedQueue<>());
counts.put(node, 0);
failedCounts.put(node, 0);
}
if (channelCount <= 0) {
throw new IllegalArgumentException("channel count must be greater zero, but got " + channelCount);
}
for (int i = 0; i < channelCount; i++) {
Channel channel = newConnection();
if (channel == null) {
throw new ConnectException("failed to prepare channels");
}
HttpAddress key = channel.attr(POOL_ATTRIBUTE_KEY).get();
if (channel.isActive()) {
Queue<Channel> channelQueue = availableChannels.get(key);
if (channelQueue != null) {
channelQueue.add(channel);
}
} else {
channel.close();
}
}
logger.log(Level.FINE,"pool: prepared " + channelCount + " channels: " + availableChannels);
}
@Override
public HttpVersion getVersion() {
return httpVersion;
}
@Override
public Channel acquire() throws Exception {
Channel channel = null;
if (semaphore.tryAcquire()) {
if ((channel = poll()) == null) {
channel = newConnection();
}
if (channel == null) {
semaphore.release();
throw new ConnectException();
} else {
if (channelPoolhandler != null) {
channelPoolhandler.channelAcquired(channel);
}
}
}
return channel;
}
@Override
public void release(Channel channel, boolean close) throws Exception {
try {
if (channel != null) {
if (channel.isActive()) {
HttpAddress key = channel.attr(POOL_ATTRIBUTE_KEY).get();
if (key != null) {
Queue<Channel> channelQueue = availableChannels.get(key);
if (channelQueue != null) {
channelQueue.add(channel);
}
}
} else if (channel.isOpen() && close) {
logger.log(Level.FINE, "closing channel " + channel);
channel.close();
}
if (channelPoolhandler != null) {
channelPoolhandler.channelReleased(channel);
}
}
} finally {
semaphore.release();
}
}
@Override
public void close() throws IOException {
lock.lock();
try {
logger.log(Level.FINE, "closing pool");
int count = 0;
Set<Channel> channelSet = new HashSet<>();
for (Map.Entry<HttpAddress, Queue<Channel>> entry : availableChannels.entrySet()) {
channelSet.addAll(entry.getValue());
}
for (Map.Entry<HttpAddress, List<Channel>> entry : channels.entrySet()) {
channelSet.addAll(entry.getValue());
}
for (Channel channel : channelSet) {
if (channel != null && channel.isOpen()) {
logger.log(Level.FINE, "trying to abort channel " + channel);
if (httpVersion.majorVersion() == 2) {
// be polite, send a go away frame
DefaultHttp2GoAwayFrame goAwayFrame = new DefaultHttp2GoAwayFrame(0);
ChannelPromise channelPromise = channel.newPromise();
channel.writeAndFlush(goAwayFrame, channelPromise);
try {
channelPromise.get();
logger.log(Level.FINE, "goaway frame sent to " + channel);
} catch (ExecutionException e) {
logger.log(Level.FINE, e.getMessage(), e);
} catch (InterruptedException e) {
throw new IOException(e);
}
}
channel.close();
count++;
}
}
availableChannels.clear();
channels.clear();
bootstraps.clear();
counts.clear();
logger.log(Level.FINE, "closed pool (found " + count + " connections open)");
} finally {
lock.unlock();
}
}
private Channel newConnection() throws ConnectException {
Channel channel = null;
HttpAddress key = null;
int min = Integer.MAX_VALUE;
Integer next;
for (int j = 0; j < numberOfNodes; j++) {
HttpAddress nextKey = poolKeySelector.key();
next = counts.get(nextKey);
if (next == null || next == 0) {
key = nextKey;
break;
} else if (next < min) {
min = next;
key = nextKey;
}
}
if (key != null) {
logger.log(Level.FINE, "trying connection to " + key);
try {
channel = connect(key);
} catch (Exception e) {
logger.log(Level.WARNING, "failed to create a new connection to " + key + ": " + e.toString());
if (retriesPerNode > 0) {
int selectedNodeFailedConnAttemptsCount = failedCounts.get(key) + 1;
failedCounts.put(key, selectedNodeFailedConnAttemptsCount);
if (selectedNodeFailedConnAttemptsCount > retriesPerNode) {
logger.log(Level.WARNING, "failed to connect to the node " + key + " "
+ selectedNodeFailedConnAttemptsCount + " times, "
+ "excluding the node from the connection pool");
counts.put(key, Integer.MAX_VALUE);
boolean allNodesExcluded = true;
for (HttpAddress node : nodes) {
if (counts.get(node) < Integer.MAX_VALUE) {
allNodesExcluded = false;
break;
}
}
if (allNodesExcluded) {
logger.log(Level.SEVERE, "no nodes left in the connection pool");
}
}
}
if (e instanceof ConnectException) {
throw (ConnectException) e;
} else {
throw new ConnectException(e.getMessage());
}
}
}
if (channel != null) {
channel.closeFuture().addListener(new CloseChannelListener(key, channel));
channel.attr(POOL_ATTRIBUTE_KEY).set(key);
channels.computeIfAbsent(key, node -> new ArrayList<>()).add(channel);
counts.put(key, counts.get(key) + 1);
if (retriesPerNode > 0) {
failedCounts.put(key, 0);
}
}
return channel;
}
private Channel connect(HttpAddress key) throws Exception {
Bootstrap bootstrap = bootstraps.get(key);
if (bootstrap != null) {
return bootstrap.connect().sync().channel();
}
return null;
}
private Channel poll() {
Queue<Channel> channelQueue;
Channel channel;
for (int j = 0; j < numberOfNodes; j++) {
HttpAddress key = poolKeySelector.key();
channelQueue = availableChannels.get(key);
if (channelQueue != null) {
channel = channelQueue.poll();
if (channel != null && channel.isActive()) {
return channel;
}
} else {
logger.log(Level.WARNING, "what happened? channel queue is null?");
}
}
return null;
}
private interface PoolKeySelector {
HttpAddress key();
}
private class RandomPoolKeySelector implements PoolKeySelector {
@Override
public HttpAddress key() {
int r = ThreadLocalRandom.current().nextInt(numberOfNodes);
return nodes.get(r % numberOfNodes);
}
}
private class RoundRobinKeySelector implements PoolKeySelector {
int r = 0;
@Override
public HttpAddress key() {
return nodes.get(r++ % numberOfNodes);
}
}
private class CloseChannelListener implements ChannelFutureListener {
private final HttpAddress key;
private final Channel channel;
private CloseChannelListener(HttpAddress key, Channel channel) {
this.key = key;
this.channel = channel;
}
@Override
public void operationComplete(ChannelFuture future) {
logger.log(Level.FINE,"connection to " + key + " closed");
lock.lock();
try {
if (counts.containsKey(key)) {
counts.put(key, counts.get(key) - 1);
}
List<Channel> channels = BoundedChannelPool.this.channels.get(key);
if (channels != null) {
channels.remove(channel);
}
semaphore.release();
} finally {
lock.unlock();
}
}
}
static class HttpChannelPoolInitializer extends ChannelInitializer<SocketChannel> {
private final HttpAddress key;
private final ChannelPoolHandler channelPoolHandler;
HttpChannelPoolInitializer(HttpAddress key, ChannelPoolHandler channelPoolHandler) {
this.key = key;
this.channelPoolHandler = channelPoolHandler;
}
@Override
protected void initChannel(SocketChannel channel) throws Exception {
if (!channel.eventLoop().inEventLoop()) {
throw new IllegalStateException();
}
channel.attr(Pool.POOL_ATTRIBUTE_KEY).set(key);
if (channelPoolHandler != null) {
channelPoolHandler.channelCreated(channel);
}
}
}
}

View file

@ -0,0 +1,13 @@
package org.xbib.net.http.client.netty;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import java.util.concurrent.ThreadFactory;
public interface ClientTransportProvider {
EventLoopGroup createEventLoopGroup(int nThreads, ThreadFactory threadFactory);
Class<? extends SocketChannel> createSocketChannelClass();
}

View file

@ -0,0 +1,20 @@
package org.xbib.net.http.client.netty;
import io.netty.channel.Channel;
import java.io.Closeable;
import java.io.IOException;
import org.xbib.net.http.HttpAddress;
public interface HttpChannelInitializer {
boolean supports(HttpAddress httpAddress);
Interaction newInteraction(NettyHttpClient client, HttpAddress httpAdress);
void init(Channel channel,
HttpAddress httpAddress,
NettyHttpClient client,
NettyCustomizer nettyCustomizer,
Interaction interaction) throws IOException;
}

View file

@ -0,0 +1,6 @@
package org.xbib.net.http.client.netty;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.pool.ChannelPoolHandler;
import io.netty.channel.socket.SocketChannel;
import org.xbib.net.http.HttpAddress;

View file

@ -0,0 +1,27 @@
package org.xbib.net.http.client.netty;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelPromise;
import io.netty.handler.codec.http.DefaultHttpContent;
import io.netty.handler.codec.http.HttpContentCompressor;
/**
* Be sure you place the HttpChunkContentCompressor before the ChunkedWriteHandler.
*/
public class HttpChunkContentCompressor extends HttpContentCompressor {
public HttpChunkContentCompressor() {
}
@Override
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
if (msg instanceof ByteBuf) {
ByteBuf byteBuf = (ByteBuf) msg;
if (byteBuf.isReadable()) {
msg = new DefaultHttpContent(byteBuf);
}
}
super.write(ctx, msg, promise);
}
}

View file

@ -0,0 +1,267 @@
package org.xbib.net.http.client.netty;
import io.netty.buffer.ByteBufAllocator;
import io.netty.buffer.PooledByteBufAllocator;
import io.netty.handler.codec.http.multipart.InterfaceHttpData;
import java.io.Closeable;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.util.Collection;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import org.xbib.net.ParameterBuilder;
import org.xbib.net.Request;
import org.xbib.net.URL;
import org.xbib.net.http.HttpHeaders;
import org.xbib.net.http.HttpMethod;
import org.xbib.net.http.HttpVersion;
import org.xbib.net.http.client.BackOff;
import org.xbib.net.http.client.ExceptionListener;
import org.xbib.net.http.client.HttpResponse;
import org.xbib.net.http.client.ResponseListener;
import org.xbib.net.http.client.TimeoutListener;
import org.xbib.net.http.cookie.Cookie;
/**
* HTTP client request.
*/
public class HttpRequest implements org.xbib.net.http.client.HttpRequest, Closeable {
private final HttpRequestBuilder builder;
private final HttpHeaders headers;
private CompletableFuture<HttpRequest> completableFuture;
private int redirectCount;
protected HttpRequest(HttpRequestBuilder builder, HttpHeaders headers) {
this.builder = builder;
this.headers = headers;
}
@Override
public URL getURL() {
return builder.url;
}
@Override
public HttpVersion getVersion() {
return builder.httpVersion;
}
@Override
public HttpMethod getMethod() {
return builder.httpMethod;
}
@Override
public HttpHeaders getHeaders() {
return headers;
}
@Override
public ParameterBuilder getParameters() {
return builder.parameterBuilder;
}
public Collection<Cookie> cookies() {
return builder.cookies;
}
@Override
public InetSocketAddress getLocalAddress() {
return null; // unused
}
@Override
public InetSocketAddress getRemoteAddress() {
return null; // unused
}
@Override
public URL getBaseURL() {
return builder.url;
}
public ByteBuffer getBody() {
return builder.body;
}
@Override
public CharBuffer getBodyAsChars(Charset charset) {
return charset.decode(getBody());
}
public CharBuffer getBodyAsChars(Charset charset, int offset, int size) {
ByteBuffer slicedBuffer = (getBody().duplicate().position(offset)).slice();
slicedBuffer.limit(size);
return charset.decode(slicedBuffer);
}
@SuppressWarnings("unchecked")
@Override
public <R extends Request> R as(Class<R> cl) {
return (R) this;
}
public List<InterfaceHttpData> getBodyData() {
return builder.bodyData;
}
public boolean isFollowRedirect() {
return builder.followRedirect;
}
public boolean isBackOff() {
return builder.backOff != null;
}
public BackOff getBackOff() {
return builder.backOff;
}
public boolean canRedirect() {
if (!builder.followRedirect) {
return false;
}
if (redirectCount >= builder.maxRedirects) {
return false;
}
redirectCount++;
return true;
}
public void release() {
// nothing to do
}
@Override
public void close() throws IOException {
release();
}
@Override
public String toString() {
return "HttpNettyRequest[url=" + builder.url +
",version=" + builder.httpVersion +
",method=" + builder.httpMethod +
",headers=" + headers.entries() +
",content=" + (builder.body != null && builder.body.remaining() >= 16 ?
getBodyAsChars(StandardCharsets.UTF_8, 0, 16) + "..." :
builder.body != null ? getBodyAsChars(StandardCharsets.UTF_8) : "") +
"]";
}
public HttpRequest setCompletableFuture(CompletableFuture<HttpRequest> completableFuture) {
this.completableFuture = completableFuture;
return this;
}
public CompletableFuture<HttpRequest> getCompletableFuture() {
return completableFuture;
}
public void setResponseListener(ResponseListener<HttpResponse> responseListener) {
builder.responseListener = responseListener;
}
public void onResponse(HttpResponse httpResponse) {
if (builder.responseListener != null) {
builder.responseListener.onResponse(httpResponse);
}
if (completableFuture != null) {
completableFuture.complete(this);
}
}
public void setExceptionListener(ExceptionListener exceptionListener) {
builder.exceptionListener = exceptionListener;
}
public void onException(Throwable throwable) {
if (builder.exceptionListener != null) {
builder.exceptionListener.onException(throwable);
}
if (completableFuture != null) {
completableFuture.completeExceptionally(throwable);
}
}
public void setTimeoutListener(TimeoutListener timeoutListener) {
builder.timeoutListener = timeoutListener;
}
public void onTimeout() {
if (builder.timeoutListener != null) {
builder.timeoutListener.onTimeout(this);
}
if (completableFuture != null) {
if (builder.timeoutMillis > 0L) {
completableFuture.completeOnTimeout(this, builder.timeoutMillis, TimeUnit.MILLISECONDS);
} else {
completableFuture.completeOnTimeout(this, 15L, TimeUnit.SECONDS);
}
}
}
public static HttpRequestBuilder get() {
return builder(HttpMethod.GET);
}
public static HttpRequestBuilder put() {
return builder(HttpMethod.PUT);
}
public static HttpRequestBuilder post() {
return builder(HttpMethod.POST);
}
public static HttpRequestBuilder delete() {
return builder(HttpMethod.DELETE);
}
public static HttpRequestBuilder head() {
return builder(HttpMethod.HEAD);
}
public static HttpRequestBuilder patch() {
return builder(HttpMethod.PATCH);
}
public static HttpRequestBuilder trace() {
return builder(HttpMethod.TRACE);
}
public static HttpRequestBuilder options() {
return builder(HttpMethod.OPTIONS);
}
public static HttpRequestBuilder connect() {
return builder(HttpMethod.CONNECT);
}
public static HttpRequestBuilder builder(HttpMethod httpMethod) {
return builder(PooledByteBufAllocator.DEFAULT, httpMethod);
}
public static HttpRequestBuilder builder(HttpMethod httpMethod, HttpRequest httpRequest) {
return builder(PooledByteBufAllocator.DEFAULT, httpMethod)
.setVersion(httpRequest.builder.httpVersion)
.setURL(httpRequest.builder.url)
.setHeaders(httpRequest.headers)
.content(httpRequest.builder.body)
.setResponseListener(httpRequest.builder.responseListener)
.setTimeoutListener(httpRequest.builder.timeoutListener, httpRequest.builder.timeoutMillis)
.setExceptionListener(httpRequest.builder.exceptionListener);
}
public static HttpRequestBuilder builder(ByteBufAllocator allocator, HttpMethod httpMethod) {
return new HttpRequestBuilder(allocator).setMethod(httpMethod);
}
}

View file

@ -0,0 +1,438 @@
package org.xbib.net.http.client.netty;
import io.netty.buffer.ByteBufAllocator;
import io.netty.handler.codec.http.HttpUtil;
import io.netty.handler.codec.http.multipart.InterfaceHttpData;
import io.netty.handler.codec.http2.HttpConversionUtil;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.nio.ByteBuffer;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.time.ZoneOffset;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Collection;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import org.xbib.net.Parameter;
import org.xbib.net.ParameterBuilder;
import org.xbib.net.URL;
import org.xbib.net.URLBuilder;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpHeaderNames;
import org.xbib.net.http.HttpHeaderValues;
import org.xbib.net.http.HttpHeaders;
import org.xbib.net.http.HttpMethod;
import org.xbib.net.http.HttpVersion;
import org.xbib.net.http.client.BackOff;
import org.xbib.net.http.client.ExceptionListener;
import org.xbib.net.http.client.HttpResponse;
import org.xbib.net.http.client.ResponseListener;
import org.xbib.net.http.client.TimeoutListener;
import org.xbib.net.http.cookie.Cookie;
public class HttpRequestBuilder implements org.xbib.net.http.client.HttpRequestBuilder {
private static final URL DEFAULT_URL = URL.from("http://localhost");
private static final String DEFAULT_FORM_CONTENT_TYPE = "application/x-www-form-urlencoded; charset=utf-8";
final ByteBufAllocator allocator;
HttpAddress httpAddress;
URL url;
String requestPath;
final Collection<Cookie> cookies;
HttpMethod httpMethod;
HttpHeaders headers;
HttpVersion httpVersion;
final List<String> removeHeaders;
String userAgent;
boolean keepalive;
boolean gzip;
String contentType;
ParameterBuilder parameterBuilder;
ByteBuffer body;
final List<InterfaceHttpData> bodyData;
boolean followRedirect;
int maxRedirects;
boolean enableBackOff;
BackOff backOff;
ResponseListener<HttpResponse> responseListener;
ExceptionListener exceptionListener;
TimeoutListener timeoutListener;
long timeoutMillis;
protected HttpRequestBuilder() {
this(ByteBufAllocator.DEFAULT);
}
protected HttpRequestBuilder(ByteBufAllocator allocator) {
this.allocator = allocator;
this.httpMethod = HttpMethod.GET;
this.httpVersion = HttpVersion.HTTP_1_1;
this.userAgent = UserAgent.getUserAgent();
this.gzip = false;
this.keepalive = true;
this.url = DEFAULT_URL;
this.followRedirect = true;
this.maxRedirects = 10;
this.headers = new HttpHeaders();
this.removeHeaders = new ArrayList<>();
this.cookies = new HashSet<>();
this.bodyData = new ArrayList<>();
this.contentType = DEFAULT_FORM_CONTENT_TYPE;
this.parameterBuilder = Parameter.builder();
this.timeoutMillis = 0L;
}
@Override
public HttpRequestBuilder setAddress(HttpAddress httpAddress) {
this.httpAddress = httpAddress;
try {
this.url = URL.builder()
.scheme(httpAddress.isSecure() ? "https" : "http")
.host(httpAddress.getInetSocketAddress().getHostString())
.port(httpAddress.getInetSocketAddress().getPort())
.build();
} catch (IOException e) {
throw new UncheckedIOException(e);
}
this.httpVersion = httpAddress.getVersion();
return this;
}
public HttpRequestBuilder setURL(String url) {
return setURL(URL.from(url));
}
@Override
public HttpRequestBuilder setURL(URL url) {
this.url = url;
return this;
}
@Override
public HttpRequestBuilder setRequestPath(String requestPath) {
this.requestPath = requestPath;
return this;
}
public HttpRequestBuilder setMethod(HttpMethod httpMethod) {
this.httpMethod = httpMethod;
return this;
}
public HttpRequestBuilder setVersion(HttpVersion httpVersion) {
this.httpVersion = httpVersion;
return this;
}
public HttpRequestBuilder setVersion(String httpVersion) {
this.httpVersion = HttpVersion.valueOf(httpVersion);
return this;
}
public HttpRequestBuilder setHeaders(Map<String, String> headers) {
headers.forEach(this::addHeader);
return this;
}
public HttpRequestBuilder setHeaders(HttpHeaders headers) {
this.headers = headers;
return this;
}
public HttpRequestBuilder addHeader(String name, String value) {
this.headers.add(name, value);
return this;
}
public HttpRequestBuilder setHeader(String name, String value) {
this.headers.set(name, value);
return this;
}
public HttpRequestBuilder removeHeader(String name) {
removeHeaders.add(name);
return this;
}
public HttpRequestBuilder contentType(String contentType) {
Objects.requireNonNull(contentType);
this.contentType = contentType;
addHeader(HttpHeaderNames.CONTENT_TYPE, contentType);
return this;
}
public HttpRequestBuilder contentType(String contentType, Charset charset) {
Objects.requireNonNull(contentType);
Objects.requireNonNull(charset);
this.contentType = contentType;
addHeader(HttpHeaderNames.CONTENT_TYPE, contentType + "; charset=" + charset.name().toLowerCase());
return this;
}
@Override
public HttpRequestBuilder setParameterBuilder(ParameterBuilder parameterBuilder) {
this.parameterBuilder = parameterBuilder;
return this;
}
public HttpRequestBuilder setParameters(Map<String, Object> parameters) {
parameters.forEach(this::addParameter);
return this;
}
@SuppressWarnings("unchecked")
public HttpRequestBuilder addParameter(String name, Object value) {
Objects.requireNonNull(name);
Objects.requireNonNull(value);
Collection<Object> collection;
if (!(value instanceof Collection)) {
collection = Collections.singletonList(value);
} else {
collection = (Collection<Object>) value;
}
collection.forEach(v -> parameterBuilder.add(name, v));
return this;
}
public HttpRequestBuilder addRawParameter(String name, String value) {
Objects.requireNonNull(name);
Objects.requireNonNull(value);
parameterBuilder.add(name, value);
return this;
}
public HttpRequestBuilder addBasicAuthorization(String name, String password) {
String encoding = Base64.getEncoder().encodeToString((name + ":" + password).getBytes(StandardCharsets.UTF_8));
this.headers.add(HttpHeaderNames.AUTHORIZATION, "Basic " + encoding);
return this;
}
@Override
public HttpRequestBuilder setBody(ByteBuffer byteBuffer) {
this.body = byteBuffer;
return this;
}
/**
* For multipart MIME body data.
*
* @param data a mime body
* @return this
*/
public HttpRequestBuilder addBodyData(InterfaceHttpData data) {
bodyData.add(data);
return this;
}
public HttpRequestBuilder addCookie(Cookie cookie) {
cookies.add(cookie);
return this;
}
public HttpRequestBuilder acceptGzip(boolean gzip) {
this.gzip = gzip;
return this;
}
public HttpRequestBuilder keepAlive(boolean keepalive) {
this.keepalive = keepalive;
return this;
}
public HttpRequestBuilder setFollowRedirect(boolean followRedirect) {
this.followRedirect = followRedirect;
return this;
}
public HttpRequestBuilder setMaxRedirects(int maxRedirects) {
this.maxRedirects = maxRedirects;
return this;
}
public HttpRequestBuilder enableBackOff(boolean enableBackOff) {
this.enableBackOff = enableBackOff;
return this;
}
public HttpRequestBuilder setBackOff(BackOff backOff) {
this.backOff = backOff;
return this;
}
public HttpRequestBuilder setUserAgent(String userAgent) {
this.userAgent = userAgent;
return this;
}
public HttpRequestBuilder text(String text) {
if (text == null) {
return this;
}
ByteBuffer byteBuf = StandardCharsets.UTF_8.encode(text);
content(byteBuf, HttpHeaderValues.TEXT_PLAIN);
return this;
}
public HttpRequestBuilder json(String json) {
if (json == null) {
return this;
}
ByteBuffer byteBuf = StandardCharsets.UTF_8.encode(json);
content(byteBuf, HttpHeaderValues.APPLICATION_JSON);
return this;
}
public HttpRequestBuilder xml(String xml) {
if (xml == null) {
return this;
}
ByteBuffer byteBuf = StandardCharsets.UTF_8.encode(xml);
content(byteBuf, "application/xml");
return this;
}
public HttpRequestBuilder content(CharSequence charSequence, CharSequence contentType) {
if (charSequence == null) {
return this;
}
content(charSequence.toString().getBytes(HttpUtil.getCharset(contentType, StandardCharsets.UTF_8)), contentType.toString());
return this;
}
public HttpRequestBuilder content(CharSequence charSequence, CharSequence contentType, Charset charset) {
if (charSequence == null) {
return this;
}
content(charSequence.toString().getBytes(charset), contentType.toString());
return this;
}
public HttpRequestBuilder content(byte[] buf, String contentType) {
if (buf == null) {
return this;
}
content(ByteBuffer.wrap(buf), contentType);
return this;
}
public HttpRequestBuilder content(ByteBuffer content, String contentType) {
if (content == null) {
return this;
}
setBody(content);
addHeader(HttpHeaderNames.CONTENT_LENGTH, Long.toString(content.remaining()));
addHeader(HttpHeaderNames.CONTENT_TYPE, contentType);
return this;
}
public HttpRequestBuilder content(ByteBuffer content) {
if (content == null) {
return this;
}
this.body = content;
return this;
}
public HttpRequestBuilder setResponseListener(ResponseListener<HttpResponse> responseListener) {
this.responseListener = responseListener;
return this;
}
public HttpRequestBuilder setExceptionListener(ExceptionListener exceptionListener) {
this.exceptionListener = exceptionListener;
return this;
}
public HttpRequestBuilder setTimeoutListener(TimeoutListener timeoutListener, long timeoutMillis) {
this.timeoutListener = timeoutListener;
this.timeoutMillis = timeoutMillis;
return this;
}
public HttpRequest build() {
return new HttpRequest(this, validateHeaders());
}
protected HttpHeaders validateHeaders() {
Parameter parameter = parameterBuilder.build();
HttpHeaders validatedHeaders = HttpHeaders.of(headers);
if (url != null) {
// add our URI parameters to the URL
URLBuilder urlBuilder = url.mutator();
if (requestPath != null) {
urlBuilder.path(requestPath);
}
parameter.forEach(e -> urlBuilder.queryParam(e.getKey(), e.getValue()));
url = urlBuilder.build();
String scheme = url.getScheme();
if (httpVersion.majorVersion() == 2) {
validatedHeaders.set(HttpConversionUtil.ExtensionHeaderNames.SCHEME.text(), scheme);
}
validatedHeaders.set(HttpHeaderNames.HOST, url.getHostInfo());
}
validatedHeaders.set(HttpHeaderNames.DATE, DateTimeFormatter.RFC_1123_DATE_TIME.format(ZonedDateTime.now(ZoneOffset.UTC)));
if (userAgent != null) {
validatedHeaders.set(HttpHeaderNames.USER_AGENT, userAgent);
}
if (gzip) {
validatedHeaders.set(HttpHeaderNames.ACCEPT_ENCODING, "gzip");
}
if (httpMethod.name().equals(HttpMethod.POST.name())) {
content(parameter.getAsQueryString(), contentType);
}
int length = body != null ? body.remaining() : 0;
if (!validatedHeaders.containsHeader(HttpHeaderNames.CONTENT_LENGTH) && !validatedHeaders.containsHeader(HttpHeaderNames.TRANSFER_ENCODING)) {
if (length < 0) {
validatedHeaders.set(HttpHeaderNames.TRANSFER_ENCODING, "chunked");
} else {
validatedHeaders.set(HttpHeaderNames.CONTENT_LENGTH, Long.toString(length));
}
}
if (!validatedHeaders.containsHeader(HttpHeaderNames.ACCEPT)) {
validatedHeaders.set(HttpHeaderNames.ACCEPT, "*/*");
}
// RFC 2616 Section 14.10
// "An HTTP/1.1 client that does not support persistent connections MUST include the "close" connection
// option in every request message."
if (httpVersion.majorVersion() == 1 && !keepalive) {
validatedHeaders.set(HttpHeaderNames.CONNECTION, "close");
}
// at last, forced removal of unwanted headers
for (String headerName : removeHeaders) {
validatedHeaders.remove(headerName);
}
return validatedHeaders;
}
}

View file

@ -0,0 +1,81 @@
package org.xbib.net.http.client.netty;
import java.io.Closeable;
import java.io.IOException;
import java.io.InputStream;
import java.net.SocketAddress;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.Charset;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpHeaders;
import org.xbib.net.http.HttpResponseStatus;
import org.xbib.net.http.cookie.CookieBox;
import org.xbib.net.util.ByteBufferInputStream;
public class HttpResponse implements org.xbib.net.http.client.HttpResponse, Closeable {
private final HttpResponseBuilder builder;
protected HttpResponse(HttpResponseBuilder builder) {
this.builder = builder;
}
public static HttpResponseBuilder builder() {
return new HttpResponseBuilder();
}
public SocketAddress getLocalAddress() {
return builder.localAddress;
}
public SocketAddress getRemoteAddress() {
return builder.remoteAddress;
}
@Override
public HttpAddress getAddress() {
return builder.httpAddress;
}
@Override
public HttpResponseStatus getStatus() {
return builder.httpStatus;
}
@Override
public HttpHeaders getHeaders() {
return builder.httpHeaders;
}
@Override
public CookieBox getCookies() {
return builder.cookieBox;
}
@Override
public ByteBuffer getBody() {
return builder.byteBuffer;
}
@Override
public CharBuffer getBodyAsChars(Charset charset) {
return charset.decode(builder.byteBuffer);
}
@Override
public InputStream getBodyAsStream() {
return new ByteBufferInputStream(builder.byteBuffer);
}
@Override
public void release() {
// nothing to do
}
@Override
public void close() throws IOException {
release();
}
}

View file

@ -0,0 +1,73 @@
package org.xbib.net.http.client.netty;
import java.io.InputStream;
import java.net.SocketAddress;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpHeaders;
import org.xbib.net.http.HttpResponseStatus;
import org.xbib.net.http.cookie.CookieBox;
public class HttpResponseBuilder {
SocketAddress localAddress;
SocketAddress remoteAddress;
HttpAddress httpAddress;
HttpResponseStatus httpStatus;
HttpHeaders httpHeaders;
CookieBox cookieBox;
ByteBuffer byteBuffer;
CharBuffer charBuffer;
InputStream inputStream;
protected HttpResponseBuilder() {
}
public HttpResponseBuilder setLocalAddress(SocketAddress localAddress) {
this.localAddress = localAddress;
return this;
}
public HttpResponseBuilder setRemoteAddress(SocketAddress remoteAddress) {
this.remoteAddress = remoteAddress;
return this;
}
public HttpResponseBuilder setHttpAddress(HttpAddress httpAddress) {
this.httpAddress = httpAddress;
return this;
}
public HttpResponseBuilder setStatus(HttpResponseStatus httpResponseStatus) {
this.httpStatus = httpResponseStatus;
return this;
}
public HttpResponseBuilder setCookieBox(CookieBox cookieBox) {
this.cookieBox = cookieBox;
return this;
}
public HttpResponseBuilder setHeaders(HttpHeaders httpHeaders) {
this.httpHeaders = httpHeaders;
return this;
}
public HttpResponseBuilder setByteBuffer(ByteBuffer byteBuffer) {
this.byteBuffer = byteBuffer;
return this;
}
public HttpResponse build() {
return new HttpResponse(this);
}
}

View file

@ -0,0 +1,61 @@
package org.xbib.net.http.client.netty;
import io.netty.channel.Channel;
import io.netty.channel.ChannelPromise;
import io.netty.handler.codec.http.FullHttpResponse;
import io.netty.handler.codec.http2.Http2Headers;
import io.netty.handler.codec.http2.Http2Settings;
import java.io.Closeable;
import java.io.IOException;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.function.Function;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.client.HttpResponse;
import org.xbib.net.http.cookie.CookieBox;
public interface Interaction extends Closeable {
HttpAddress getHttpAddress();
Interaction execute(HttpRequest httpRequest) throws IOException;
<T> CompletableFuture<T> execute(HttpRequest httpRequest, Function<HttpResponse, T> supplier) throws IOException;
void settingsPrefaceWritten() throws IOException;
void setSettingsPromise(ChannelPromise channelPromise);
void waitForSettings(long value, TimeUnit timeUnit) throws ExecutionException, InterruptedException, TimeoutException;
void settingsReceived(Http2Settings http2Settings) throws IOException;
void responseReceived(Channel channel, Integer streamId, FullHttpResponse fullHttpResponse) throws IOException;
void pushPromiseReceived(Channel channel, Integer streamId, Integer promisedStreamId, Http2Headers headers);
void fail(Channel channel, Throwable throwable);
void inactive(Channel channel);
void setCookieBox(CookieBox cookieBox);
CookieBox getCookieBox();
void setFuture(CompletableFuture<?> future);
CompletableFuture<?> getFuture();
Interaction get();
Interaction get(long value, TimeUnit timeUnit);
void cancel();
boolean isFailed();
Throwable getFailure();
}

View file

@ -0,0 +1,46 @@
package org.xbib.net.http.client.netty;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.http.FullHttpRequest;
/**
* Strategy interface to customize netty {@link Bootstrap} and {@link Channel} via callback hooks.
* <b>Extending the NettyCustomizer API</b>
* Contrary to other driver options, the options available in this class should be considered as advanced feature and as such,
* they should only be modified by expert users. A misconfiguration introduced by the means of this API can have unexpected
* results and cause the driver to completely fail to connect.
*/
public interface NettyCustomizer {
/**
* Hook invoked each time the driver creates a new Connection and configures a new instance of Bootstrap for it. This hook
* is called after the driver has applied all {@link java.net.SocketOption}s. This is a good place to add extra
* {@link io.netty.channel.ChannelOption}s to the {@link Bootstrap}.
*
* @param bootstrap must not be {@code null}.
*/
default void afterBootstrapInitialized(Bootstrap bootstrap) {
}
/**
* Hook invoked each time the driver initializes the channel. This hook is called after the driver has registered all its
* internal channel handlers, and applied the configured options.
*
* @param channel must not be {@code null}.
*/
default void afterChannelInitialized(Channel channel) {
}
/**
* Hook invoked each time a full HTTP request is received in a Netty handler pipeline.
* Useful to adjust headers in a Netty way.
*
* @param ctx the channel context
* @param fullHttpRequest the full HTTP request
*/
default void afterFullHttpRequestReceived(ChannelHandlerContext ctx, FullHttpRequest fullHttpRequest) {
}
}

View file

@ -0,0 +1,260 @@
package org.xbib.net.http.client.netty;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.pool.ChannelPoolHandler;
import io.netty.util.concurrent.Future;
import java.io.Closeable;
import java.io.IOException;
import java.net.ConnectException;
import java.util.List;
import java.util.ServiceLoader;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.Function;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.client.HttpClient;
import org.xbib.net.http.client.HttpResponse;
public class NettyHttpClient implements HttpClient<HttpRequest, HttpResponse>, Closeable {
private static final Logger logger = Logger.getLogger(NettyHttpClient.class.getName());
private final NettyHttpClientBuilder builder;
private final EventLoopGroup eventLoopGroup;
private final Bootstrap bootstrap;
private final AtomicBoolean closed;
private final HttpChannelInitializer httpChannelInitializer;
private final ServiceLoader<HttpChannelInitializer> httpChannelInitializerServiceLoader;
private Pool pool;
private final List<Interaction> interactions;
NettyHttpClient(NettyHttpClientBuilder builder,
EventLoopGroup eventLoopGroup,
Bootstrap bootstrap) throws IOException {
this.builder = builder;
this.eventLoopGroup = eventLoopGroup;
this.bootstrap = bootstrap;
this.closed = new AtomicBoolean(false);
this.httpChannelInitializer = builder.httpChannelInitializer;
this.httpChannelInitializerServiceLoader = ServiceLoader.load(HttpChannelInitializer.class);
createBoundedPool(builder.nettyHttpClientConfig, bootstrap);
this.interactions = new CopyOnWriteArrayList<>();
}
public static NettyHttpClientBuilder builder() {
return new NettyHttpClientBuilder();
}
public NettyHttpClient getClient() {
return this;
}
public Bootstrap getBootstrap() {
return bootstrap;
}
public NettyHttpClientConfig getClientConfig() {
return builder.nettyHttpClientConfig;
}
public Pool getPool() {
return pool;
}
public boolean hasPooledNodes() {
return pool != null && !builder.nettyHttpClientConfig.getPoolNodes().isEmpty();
}
public ChannelInitializer<Channel> newChannelInitializer(HttpAddress httpAddress, Interaction interaction) {
return new ChannelInitializer<>() {
@Override
protected void initChannel(Channel channel) throws Exception {
interaction.setSettingsPromise(channel.newPromise());
lookupChannelInitializer(httpAddress)
.init(channel, httpAddress, getClient(), builder.nettyCustomizer, interaction);
}
};
}
/**
* Execute a HTTP request and return a {@link CompletableFuture}.
*
* @param request the request
* @param supplier the function for the response
* @param <T> the result of the function for the response
* @return the completable future
* @throws IOException if the request fails to be executed.
*/
@Override
public <T> CompletableFuture<T> execute(HttpRequest request,
Function<HttpResponse, T> supplier) throws IOException {
HttpAddress httpAddress = HttpAddress.of(request.getURL(), request.getVersion());
HttpChannelInitializer initializer = lookupChannelInitializer(httpAddress);
Interaction interaction = initializer.newInteraction(this, httpAddress);
interactions.add(interaction);
return interaction.execute(request, supplier);
}
/**
* Execute HTTP request.
*
* @param request the HTTP request
* @return an interaction
* @throws IOException if execution fails
*/
public Interaction execute(HttpRequest request) throws IOException {
HttpAddress httpAddress = HttpAddress.of(request.getURL(), request.getVersion());
HttpChannelInitializer initializer = lookupChannelInitializer(httpAddress);
Interaction interaction = initializer.newInteraction(this, httpAddress);
CompletableFuture<?> future = new CompletableFuture<>();
interaction.setFuture(future);
interactions.add(interaction);
return interaction.execute(request);
}
/**
* For following redirects, construct a new interaction on a given request URL..
*
* @param interaction the previous interaction
* @param request the new request for continuing the request.
* @throws IOException if continuation fails
*/
public void continuation(Interaction interaction, HttpRequest request) throws IOException {
HttpAddress httpAddress = HttpAddress.of(request.getURL(), request.getVersion());
HttpChannelInitializer initializer = lookupChannelInitializer(httpAddress);
Interaction next = initializer.newInteraction(this, httpAddress);
next.setCookieBox(interaction.getCookieBox());
next.execute(request);
next.get();
closeAndRemove(next);
}
/**
* Retry interaction.
*
* @param interaction the interaction to retry
* @param request the request to retry
* @throws IOException if retry failed
*/
public void retry(Interaction interaction, HttpRequest request) throws IOException {
interaction.execute(request);
interaction.get();
closeAndRemove(interaction);
}
@Override
public void close() throws IOException {
long amount = 15;
TimeUnit timeUnit = TimeUnit.SECONDS;
if (closed.compareAndSet(false, true)) {
try {
for (Interaction interaction : interactions) {
logger.log(Level.FINER, "waiting for unfinshed interaction " + interaction);
//interaction.get();
interaction.close();
}
if (hasPooledNodes()) {
logger.log(Level.FINER, "closing pool");
pool.close();
}
Future<?> future = eventLoopGroup.shutdownGracefully(0L, amount, timeUnit);
future.await(amount, timeUnit);
if (future.isSuccess()) {
logger.log(Level.FINER, "event loop group closed");
} else {
logger.log(Level.WARNING, "timeout when closing event loop group");
}
} catch (Exception e) {
throw new IOException(e);
}
}
}
private void closeAndRemove(Interaction interaction) {
try {
interaction.close();
remove(interaction);
} catch (Exception e) {
logger.log(Level.SEVERE, "unable to close interaction: " + e.getMessage(), e);
}
}
void remove(Interaction interaction) {
interactions.remove(interaction);
}
private HttpChannelInitializer lookupChannelInitializer(HttpAddress httpAddress) {
if (httpChannelInitializer != null || httpAddress == null) {
return httpChannelInitializer;
}
for (HttpChannelInitializer initializer : httpChannelInitializerServiceLoader) {
if (initializer.supports(httpAddress)) {
return initializer;
}
}
throw new IllegalStateException("no channel initializer found for address " + httpAddress + ", check service provider");
}
private void createBoundedPool(NettyHttpClientConfig nettyHttpClientConfig,
Bootstrap bootstrap) throws IOException {
List<HttpAddress> nodes = nettyHttpClientConfig.getPoolNodes();
if (nodes == null || nodes.isEmpty()) {
return;
}
Integer limit = nettyHttpClientConfig.getPoolNodeConnectionLimit();
if (limit == null || limit < 1) {
limit = 1;
}
Semaphore semaphore = new Semaphore(limit);
Integer retries = nettyHttpClientConfig.getRetriesPerPoolNode();
if (retries == null || retries < 0) {
retries = 0;
}
Integer nodeConnectionLimit = nettyHttpClientConfig.getPoolNodeConnectionLimit();
if (nodeConnectionLimit == null || nodeConnectionLimit == 0) {
nodeConnectionLimit = nodes.size();
}
this.pool = new BoundedChannelPool(semaphore, nettyHttpClientConfig.getPoolVersion(),
nodes, retries, nettyHttpClientConfig.getPoolKeySelectorType());
try {
this.pool.init(bootstrap, new NettyClientChannelPoolHandler(), nodeConnectionLimit);
} catch (ConnectException e) {
logger.log(Level.WARNING, e.getMessage(), e);
}
}
private class NettyClientChannelPoolHandler implements ChannelPoolHandler {
@Override
public void channelReleased(Channel channel) {
}
@Override
public void channelAcquired(Channel channel) {
}
@Override
public void channelCreated(Channel channel) throws IOException {
HttpAddress httpAddress = channel.attr(Pool.POOL_ATTRIBUTE_KEY).get();
HttpChannelInitializer initializer = lookupChannelInitializer(httpAddress);
Interaction interaction = initializer.newInteraction(getClient(), httpAddress);
initializer.init(channel, httpAddress, getClient(), builder.nettyCustomizer, interaction);
}
}
}

View file

@ -0,0 +1,177 @@
package org.xbib.net.http.client.netty;
import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.ByteBufAllocator;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import java.io.IOException;
import java.security.Provider;
import java.security.Security;
import java.util.Arrays;
import java.util.ServiceLoader;
import java.util.concurrent.ThreadFactory;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.stream.Collectors;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.util.NamedThreadFactory;
public class NettyHttpClientBuilder {
private static final Logger logger = Logger.getLogger(NettyHttpClientBuilder.class.getName());
NettyHttpClientConfig nettyHttpClientConfig;
ByteBufAllocator byteBufAllocator;
EventLoopGroup eventLoopGroup;
Class<? extends SocketChannel> socketChannelClass;
HttpChannelInitializer httpChannelInitializer;
NettyCustomizer nettyCustomizer;
NettyHttpClientBuilder() {
}
public NettyHttpClientBuilder setConfig(NettyHttpClientConfig nettyHttpClientConfig) {
this.nettyHttpClientConfig = nettyHttpClientConfig;
return this;
}
/**
* Set Netty's ByteBuf allocator.
*
* @param byteBufAllocator the byte buf allocator
* @return this builder
*/
public NettyHttpClientBuilder setByteBufAllocator(ByteBufAllocator byteBufAllocator) {
this.byteBufAllocator = byteBufAllocator;
return this;
}
public NettyHttpClientBuilder setEventLoop(EventLoopGroup eventLoopGroup) {
this.eventLoopGroup = eventLoopGroup;
return this;
}
public NettyHttpClientBuilder setChannelClass(Class<SocketChannel> socketChannelClass) {
this.socketChannelClass = socketChannelClass;
return this;
}
public NettyHttpClientBuilder addPoolNode(HttpAddress httpAddress) {
nettyHttpClientConfig.addPoolNode(httpAddress);
nettyHttpClientConfig.setPoolVersion(httpAddress.getVersion());
nettyHttpClientConfig.setPoolSecure(httpAddress.isSecure());
return this;
}
public NettyHttpClientBuilder setHttpChannelInitializer(HttpChannelInitializer httpChannelInitializer) {
this.httpChannelInitializer = httpChannelInitializer;
return this;
}
public NettyHttpClientBuilder setNettyCustomizer(NettyCustomizer nettyCustomizer) {
this.nettyCustomizer = nettyCustomizer;
return this;
}
public NettyHttpClient build() throws IOException {
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "installed security providers = " +
Arrays.stream(Security.getProviders()).map(Provider::getName).collect(Collectors.toList()));
}
if (nettyHttpClientConfig == null) {
nettyHttpClientConfig = createEmptyConfig();
}
if (byteBufAllocator == null) {
byteBufAllocator = ByteBufAllocator.DEFAULT;
}
EventLoopGroup myEventLoopGroup = createEventLoopGroup(nettyHttpClientConfig, eventLoopGroup);
Class<? extends SocketChannel> mySocketChannelClass = createChannelClass(nettyHttpClientConfig, socketChannelClass);
Bootstrap bootstrap = createBootstrap(nettyHttpClientConfig, byteBufAllocator, myEventLoopGroup, mySocketChannelClass);
if (nettyCustomizer != null) {
nettyCustomizer.afterBootstrapInitialized(bootstrap);
}
return new NettyHttpClient(this, myEventLoopGroup, bootstrap);
}
protected NettyHttpClientConfig createEmptyConfig() {
return new NettyHttpClientConfig();
}
private static EventLoopGroup createEventLoopGroup(NettyHttpClientConfig clientConfig,
EventLoopGroup eventLoopGroup) {
if (eventLoopGroup != null) {
return eventLoopGroup;
}
EventLoopGroup myEventLoopGroup = null;
ThreadFactory threadFactory = new NamedThreadFactory("org-xbib-net-http-netty-client");
ServiceLoader<ClientTransportProvider> transportProviders = ServiceLoader.load(ClientTransportProvider.class);
for (ClientTransportProvider serverTransportProvider : transportProviders) {
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "found event loop group provider = " + serverTransportProvider);
}
if (clientConfig.getTransportProviderName() == null || clientConfig.getTransportProviderName().equals(serverTransportProvider.getClass().getName())) {
myEventLoopGroup = serverTransportProvider.createEventLoopGroup(clientConfig.getThreadCount(), threadFactory);
break;
}
}
if (myEventLoopGroup == null) {
myEventLoopGroup = new NioEventLoopGroup(clientConfig.getThreadCount(), threadFactory);
}
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "event loop group class: " + myEventLoopGroup.getClass().getName());
}
return myEventLoopGroup;
}
private static Class<? extends SocketChannel> createChannelClass(NettyHttpClientConfig clientConfig,
Class<? extends SocketChannel> socketChannelClass) {
if (socketChannelClass != null) {
return socketChannelClass;
}
Class<? extends SocketChannel> myChannelClass = null;
ServiceLoader<ClientTransportProvider> transportProviders = ServiceLoader.load(ClientTransportProvider.class);
for (ClientTransportProvider transportProvider : transportProviders) {
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "found socket channel provider = " + transportProvider);
}
if (clientConfig.getTransportProviderName() == null || clientConfig.getTransportProviderName().equals(transportProvider.getClass().getName())) {
myChannelClass = transportProvider.createSocketChannelClass();
break;
}
}
if (myChannelClass == null) {
myChannelClass = NioSocketChannel.class;
}
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "socket channel class: " + myChannelClass.getName());
}
return myChannelClass;
}
private static Bootstrap createBootstrap(NettyHttpClientConfig nettyHttpClientConfig,
ByteBufAllocator byteBufAllocator,
EventLoopGroup eventLoopGroup,
Class<? extends SocketChannel> socketChannelClass) {
return new Bootstrap()
.group(eventLoopGroup)
.channel(socketChannelClass)
.option(ChannelOption.ALLOCATOR, byteBufAllocator)
.option(ChannelOption.TCP_NODELAY, nettyHttpClientConfig.socketConfig.isTcpNodelay())
.option(ChannelOption.SO_KEEPALIVE, nettyHttpClientConfig.socketConfig.isKeepAlive())
.option(ChannelOption.SO_REUSEADDR, nettyHttpClientConfig.socketConfig.isReuseAddr())
.option(ChannelOption.SO_LINGER, nettyHttpClientConfig.socketConfig.getLinger())
.option(ChannelOption.SO_SNDBUF, nettyHttpClientConfig.socketConfig.getTcpSendBufferSize())
.option(ChannelOption.SO_RCVBUF, nettyHttpClientConfig.socketConfig.getTcpReceiveBufferSize())
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, nettyHttpClientConfig.socketConfig.getConnectTimeoutMillis())
.option(ChannelOption.WRITE_BUFFER_WATER_MARK, nettyHttpClientConfig.getWriteBufferWaterMark());
}
}

View file

@ -0,0 +1,333 @@
package org.xbib.net.http.client.netty;
import io.netty.buffer.ByteBufAllocator;
import io.netty.channel.WriteBufferWaterMark;
import io.netty.handler.codec.http2.Http2Settings;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.proxy.HttpProxyHandler;
import io.netty.handler.proxy.Socks4ProxyHandler;
import io.netty.handler.proxy.Socks5ProxyHandler;
import java.util.ArrayList;
import java.util.List;
import org.xbib.net.SocketConfig;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpVersion;
import org.xbib.net.http.client.BackOff;
public class NettyHttpClientConfig {
/**
* If frame logging /traffic logging is enabled or not.
*/
private boolean debug = false;
/**
* Default debug log level.
*/
private LogLevel debugLogLevel = LogLevel.DEBUG;
SocketConfig socketConfig = new SocketConfig();
private String transportProviderName = null;
/**
* If set to 0, then Netty will decide about thread count.
* Default is Runtime.getRuntime().availableProcessors() * 2
*/
private int threadCount = 0;
/**
* Set HTTP initial line length to 4k.
* See {@link io.netty.handler.codec.http.HttpClientCodec}.
*/
private int maxInitialLineLength = 4 * 1024;
/**
* Set HTTP maximum headers size to 8k.
* See {@link io.netty.handler.codec.http.HttpClientCodec}.
*/
private int maxHeadersSize = 8 * 1024;
/**
* Set HTTP chunk maximum size to 8k.
* See {@link io.netty.handler.codec.http.HttpClientCodec}.
*/
private int maxChunkSize = 8 * 1024;
/**
* Set maximum content length to 256 MB.
*/
private int maxContentLength = 256 * 1024 * 1024;
/**
* This is Netty's default.
*/
private int maxCompositeBufferComponents = 1024;
/**
* Default for gzip codec is false
*/
private boolean gzipEnabled = false;
private ByteBufAllocator byteBufAllocator;
private HttpProxyHandler httpProxyHandler;
private Socks4ProxyHandler socks4ProxyHandler;
private Socks5ProxyHandler socks5ProxyHandler;
private List<HttpAddress> poolNodes = new ArrayList<>();
private Pool.PoolKeySelectorType poolKeySelectorType = Pool.PoolKeySelectorType.ROUNDROBIN;
private Integer poolNodeConnectionLimit;
private Integer retriesPerPoolNode = 0;
private HttpVersion poolVersion = HttpVersion.HTTP_1_1;
private Boolean poolSecure = false;
private Http2Settings http2Settings = Http2Settings.defaultSettings();
private WriteBufferWaterMark writeBufferWaterMark = WriteBufferWaterMark.DEFAULT;
private BackOff backOff = BackOff.ZERO_BACKOFF;
public NettyHttpClientConfig() {
this.byteBufAllocator = ByteBufAllocator.DEFAULT;
}
public void setByteBufAllocator(ByteBufAllocator byteBufAllocator) {
this.byteBufAllocator = byteBufAllocator;
}
public ByteBufAllocator getByteBufAllocator() {
return byteBufAllocator;
}
public NettyHttpClientConfig setDebug(boolean debug) {
this.debug = debug;
return this;
}
public NettyHttpClientConfig enableDebug() {
this.debug = true;
return this;
}
public NettyHttpClientConfig disableDebug() {
this.debug = false;
return this;
}
public boolean isDebug() {
return debug;
}
public NettyHttpClientConfig setDebugLogLevel(LogLevel debugLogLevel) {
this.debugLogLevel = debugLogLevel;
return this;
}
public LogLevel getDebugLogLevel() {
return debugLogLevel;
}
public NettyHttpClientConfig setTransportProviderName(String transportProviderName) {
this.transportProviderName = transportProviderName;
return this;
}
public String getTransportProviderName() {
return transportProviderName;
}
public NettyHttpClientConfig setThreadCount(int threadCount) {
this.threadCount = threadCount;
return this;
}
public int getThreadCount() {
return threadCount;
}
public NettyHttpClientConfig setSocketConfig(SocketConfig socketConfig) {
this.socketConfig = socketConfig;
return this;
}
public SocketConfig getSocketConfig() {
return socketConfig;
}
public NettyHttpClientConfig setMaxInitialLineLength(int maxInitialLineLength) {
this.maxInitialLineLength = maxInitialLineLength;
return this;
}
public int getMaxInitialLineLength() {
return maxInitialLineLength;
}
public NettyHttpClientConfig setMaxHeadersSize(int maxHeadersSize) {
this.maxHeadersSize = maxHeadersSize;
return this;
}
public int getMaxHeadersSize() {
return maxHeadersSize;
}
public NettyHttpClientConfig setMaxChunkSize(int maxChunkSize) {
this.maxChunkSize = maxChunkSize;
return this;
}
public int getMaxChunkSize() {
return maxChunkSize;
}
public NettyHttpClientConfig setMaxContentLength(int maxContentLength) {
this.maxContentLength = maxContentLength;
return this;
}
public int getMaxContentLength() {
return maxContentLength;
}
public NettyHttpClientConfig setMaxCompositeBufferComponents(int maxCompositeBufferComponents) {
this.maxCompositeBufferComponents = maxCompositeBufferComponents;
return this;
}
public int getMaxCompositeBufferComponents() {
return maxCompositeBufferComponents;
}
public NettyHttpClientConfig setGzipEnabled(boolean gzipEnabled) {
this.gzipEnabled = gzipEnabled;
return this;
}
public boolean isGzipEnabled() {
return gzipEnabled;
}
public NettyHttpClientConfig setHttp2Settings(Http2Settings http2Settings) {
this.http2Settings = http2Settings;
return this;
}
public Http2Settings getHttp2Settings() {
return http2Settings;
}
public NettyHttpClientConfig setHttpProxyHandler(HttpProxyHandler httpProxyHandler) {
this.httpProxyHandler = httpProxyHandler;
return this;
}
public HttpProxyHandler getHttpProxyHandler() {
return httpProxyHandler;
}
public NettyHttpClientConfig setSocks4ProxyHandler(Socks4ProxyHandler socks4ProxyHandler) {
this.socks4ProxyHandler = socks4ProxyHandler;
return this;
}
public Socks4ProxyHandler getSocks4ProxyHandler() {
return socks4ProxyHandler;
}
public NettyHttpClientConfig setSocks5ProxyHandler(Socks5ProxyHandler socks5ProxyHandler) {
this.socks5ProxyHandler = socks5ProxyHandler;
return this;
}
public Socks5ProxyHandler getSocks5ProxyHandler() {
return socks5ProxyHandler;
}
public NettyHttpClientConfig setPoolNodes(List<HttpAddress> poolNodes) {
this.poolNodes = poolNodes;
return this;
}
public List<HttpAddress> getPoolNodes() {
return poolNodes;
}
public NettyHttpClientConfig setPoolKeySelectorType(Pool.PoolKeySelectorType poolKeySelectorType) {
this.poolKeySelectorType = poolKeySelectorType;
return this;
}
public Pool.PoolKeySelectorType getPoolKeySelectorType() {
return poolKeySelectorType;
}
public NettyHttpClientConfig addPoolNode(HttpAddress poolNodeAddress) {
this.poolNodes.add(poolNodeAddress);
return this;
}
public NettyHttpClientConfig setPoolNodeConnectionLimit(Integer poolNodeConnectionLimit) {
this.poolNodeConnectionLimit = poolNodeConnectionLimit;
return this;
}
public Integer getPoolNodeConnectionLimit() {
return poolNodeConnectionLimit;
}
public NettyHttpClientConfig setRetriesPerPoolNode(Integer retriesPerPoolNode) {
this.retriesPerPoolNode = retriesPerPoolNode;
return this;
}
public Integer getRetriesPerPoolNode() {
return retriesPerPoolNode;
}
public NettyHttpClientConfig setPoolVersion(HttpVersion poolVersion) {
this.poolVersion = poolVersion;
return this;
}
public HttpVersion getPoolVersion() {
return poolVersion;
}
public NettyHttpClientConfig setPoolSecure(boolean poolSecure) {
this.poolSecure = poolSecure;
return this;
}
public boolean isPoolSecure() {
return poolSecure;
}
public NettyHttpClientConfig setWriteBufferWaterMark(WriteBufferWaterMark writeBufferWaterMark) {
this.writeBufferWaterMark = writeBufferWaterMark;
return this;
}
public WriteBufferWaterMark getWriteBufferWaterMark() {
return writeBufferWaterMark;
}
public NettyHttpClientConfig setBackOff(BackOff backOff) {
this.backOff = backOff;
return this;
}
public BackOff getBackOff() {
return backOff;
}
}

View file

@ -0,0 +1,23 @@
package org.xbib.net.http.client.netty;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import java.util.concurrent.ThreadFactory;
public class NioClientTransportProvider implements ClientTransportProvider {
public NioClientTransportProvider() {
}
@Override
public EventLoopGroup createEventLoopGroup(int nThreads, ThreadFactory threadFactory) {
return new NioEventLoopGroup(nThreads, threadFactory);
}
@Override
public Class<? extends SocketChannel> createSocketChannelClass() {
return NioSocketChannel.class;
}
}

View file

@ -0,0 +1,27 @@
package org.xbib.net.http.client.netty;
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.Channel;
import io.netty.channel.pool.ChannelPoolHandler;
import io.netty.util.AttributeKey;
import java.io.Closeable;
import java.io.IOException;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpVersion;
public interface Pool extends Closeable {
AttributeKey<HttpAddress> POOL_ATTRIBUTE_KEY = AttributeKey.valueOf("__pool");
void init(Bootstrap bootstrap, ChannelPoolHandler channelPoolHandler, int count) throws IOException;
HttpVersion getVersion();
Channel acquire() throws Exception;
void release(Channel channel, boolean close) throws Exception;
enum PoolKeySelectorType {
RANDOM, ROUNDROBIN
}
}

View file

@ -0,0 +1,71 @@
package org.xbib.net.http.client.netty;
import java.util.Set;
import java.util.SortedMap;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentSkipListMap;
import java.util.concurrent.atomic.AtomicInteger;
public class StreamIds {
private final AtomicInteger streamId;
private final SortedMap<Integer, CompletableFuture<Boolean>> sortedMap;
public StreamIds() {
this.streamId = new AtomicInteger(3);
this.sortedMap = new ConcurrentSkipListMap<>();
}
public CompletableFuture<Boolean> get(Integer key) {
return sortedMap.get(key);
}
public Set<Integer> keys() {
return sortedMap.keySet();
}
public Integer lastKey() {
return sortedMap.isEmpty() ? null : sortedMap.lastKey();
}
public void put(Integer key, CompletableFuture<Boolean> promise) {
sortedMap.put(key, promise);
}
public void remove(Integer key) {
if (key != null) {
sortedMap.remove(key);
}
}
public Integer nextStreamId() {
int streamId = this.streamId.getAndAdd(2);
if (streamId == Integer.MIN_VALUE) {
// reset if overflow, Java wraps atomic integers to Integer.MIN_VALUE
this.streamId.set(3);
streamId = 3;
}
sortedMap.put(streamId, new CompletableFuture<>());
return streamId;
}
public void fail(Throwable throwable) {
for (CompletableFuture<Boolean> promise : sortedMap.values()) {
promise.completeExceptionally(throwable);
}
}
public void close() {
sortedMap.clear();
}
public boolean isClosed() {
return sortedMap.isEmpty();
}
@Override
public String toString() {
return "StreamIds[id=" + streamId + ",map=" + sortedMap + "]";
}
}

View file

@ -0,0 +1,43 @@
package org.xbib.net.http.client.netty;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandler;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelPromise;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;
/**
* A Netty handler that logs the I/O traffic of a connection.
*/
@ChannelHandler.Sharable
public class TrafficLoggingHandler extends LoggingHandler {
public TrafficLoggingHandler(LogLevel level) {
super("client", level);
}
@Override
public void channelRegistered(ChannelHandlerContext ctx) {
ctx.fireChannelRegistered();
}
@Override
public void channelUnregistered(ChannelHandlerContext ctx) {
ctx.fireChannelUnregistered();
}
@Override
public void flush(ChannelHandlerContext ctx) {
ctx.flush();
}
@Override
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
if (msg instanceof ByteBuf && !((ByteBuf) msg).isReadable()) {
ctx.write(msg, promise);
} else {
super.write(ctx, msg, promise);
}
}
}

View file

@ -0,0 +1,43 @@
package org.xbib.net.http.client.netty;
import io.netty.bootstrap.Bootstrap;
import java.util.Optional;
/**
* HTTP client user agent.
*/
public final class UserAgent {
/**
* The default value for {@code User-Agent}.
*/
private static final String USER_AGENT = String.format("HttpNettyClient/%s (Java/%s/%s) (Netty/%s)",
httpClientVersion(), javaVendor(), javaVersion(), nettyVersion());
private UserAgent() {
}
public static String getUserAgent() {
return USER_AGENT;
}
private static String httpClientVersion() {
return Optional.ofNullable(UserAgent.class.getPackage().getImplementationVersion())
.orElse("unknown");
}
private static String javaVendor() {
return Optional.ofNullable(System.getProperty("java.vendor"))
.orElse("unknown");
}
private static String javaVersion() {
return Optional.ofNullable(System.getProperty("java.version"))
.orElse("unknown");
}
private static String nettyVersion() {
return Optional.ofNullable(Bootstrap.class.getPackage().getImplementationVersion())
.orElse("unknown");
}
}

View file

@ -0,0 +1,102 @@
package org.xbib.net.http.client.netty.http1;
import io.netty.channel.Channel;
import io.netty.channel.ChannelPipeline;
import io.netty.handler.codec.http.HttpClientCodec;
import io.netty.handler.codec.http.HttpContentDecompressor;
import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.proxy.Socks5ProxyHandler;
import io.netty.handler.stream.ChunkedWriteHandler;
import io.netty.handler.timeout.IdleStateHandler;
import io.netty.handler.timeout.ReadTimeoutHandler;
import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpVersion;
import org.xbib.net.http.client.netty.HttpChannelInitializer;
import org.xbib.net.http.client.netty.Interaction;
import org.xbib.net.http.client.netty.NettyCustomizer;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.TrafficLoggingHandler;
public class Http1ChannelInitializer implements HttpChannelInitializer {
private static final Logger logger = Logger.getLogger(Http1ChannelInitializer.class.getName());
public Http1ChannelInitializer() {
}
@Override
public boolean supports(HttpAddress httpAddress) {
return HttpVersion.HTTP_1_1.equals(httpAddress.getVersion()) && !httpAddress.isSecure();
}
@Override
public Interaction newInteraction(NettyHttpClient client, HttpAddress httpAddress) {
return new Http1Interaction(client, httpAddress);
}
@Override
public void init(Channel channel,
HttpAddress httpAddress,
NettyHttpClient nettyHttpClient,
NettyCustomizer nettyCustomizer,
Interaction interaction) throws IOException {
NettyHttpClientConfig nettyHttpClientConfig = nettyHttpClient.getClientConfig();
ChannelPipeline pipeline = channel.pipeline();
if (nettyHttpClientConfig.isDebug()) {
pipeline.addLast("client-traffic", new TrafficLoggingHandler(LogLevel.DEBUG));
}
int readTimeoutMilllis = nettyHttpClientConfig.getSocketConfig().getReadTimeoutMillis();
if (readTimeoutMilllis > 0) {
pipeline.addLast("client-read-timeout", new ReadTimeoutHandler(readTimeoutMilllis / 1000));
}
int socketTimeoutMillis = nettyHttpClientConfig.getSocketConfig().getSocketTimeoutMillis();
if (socketTimeoutMillis > 0) {
pipeline.addLast("client-idle-timeout", new IdleStateHandler(socketTimeoutMillis / 1000,
socketTimeoutMillis / 1000, socketTimeoutMillis / 1000));
}
if (nettyHttpClientConfig.getHttpProxyHandler() != null) {
pipeline.addLast("client-http-proxy", nettyHttpClientConfig.getHttpProxyHandler());
}
if (nettyHttpClientConfig.getSocks4ProxyHandler() != null) {
pipeline.addLast("client-socks4-proxy", nettyHttpClientConfig.getSocks4ProxyHandler());
}
if (nettyHttpClientConfig.getSocks5ProxyHandler() != null) {
Socks5ProxyHandler socks5ProxyHandler = nettyHttpClientConfig.getSocks5ProxyHandler();
pipeline.addLast("client-socks5-proxy", socks5ProxyHandler);
}
configurePlain(channel, nettyHttpClient, interaction);
if (nettyCustomizer != null) {
nettyCustomizer.afterChannelInitialized(channel);
}
if (nettyHttpClientConfig.isDebug()) {
logger.log(Level.FINE, "HTTP 1.1 plain channel initialized: " +
" address=" + httpAddress +
" pipeline=" + pipeline.names());
}
}
private void configurePlain(Channel channel,
NettyHttpClient nettyHttpClient,
Interaction interaction) throws IOException {
NettyHttpClientConfig nettyHttpClientConfig = nettyHttpClient.getClientConfig();
ChannelPipeline pipeline = channel.pipeline();
pipeline.addLast("http-client-chunk-writer",
new ChunkedWriteHandler());
pipeline.addLast("http-client-codec", new HttpClientCodec(nettyHttpClientConfig.getMaxInitialLineLength(),
nettyHttpClientConfig.getMaxHeadersSize(), nettyHttpClientConfig.getMaxChunkSize()));
if (nettyHttpClientConfig.isGzipEnabled()) {
pipeline.addLast("http-client-decompressor", new HttpContentDecompressor());
}
HttpObjectAggregator httpObjectAggregator =
new HttpObjectAggregator(nettyHttpClientConfig.getMaxContentLength(), false);
httpObjectAggregator.setMaxCumulationBufferComponents(nettyHttpClientConfig.getMaxCompositeBufferComponents());
pipeline.addLast("http-client-aggregator", httpObjectAggregator);
pipeline.addLast("http-client-response", new Http1Handler(interaction));
interaction.settingsReceived(null);
}
}

View file

@ -0,0 +1,47 @@
package org.xbib.net.http.client.netty.http1;
import io.netty.channel.ChannelDuplexHandler;
import io.netty.channel.ChannelHandler;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.http.FullHttpResponse;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.xbib.net.http.client.netty.Interaction;
@ChannelHandler.Sharable
public class Http1Handler extends ChannelDuplexHandler {
private static final Logger logger = Logger.getLogger(Http1Handler.class.getName());
private final Interaction interaction;
public Http1Handler(Interaction interaction) {
this.interaction = interaction;
}
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
if (msg instanceof FullHttpResponse) {
FullHttpResponse httpResponse = (FullHttpResponse) msg;
try {
interaction.responseReceived(ctx.channel(), null, httpResponse);
} finally {
httpResponse.release();
}
} else {
super.channelRead(ctx, msg);
}
}
@Override
public void userEventTriggered(ChannelHandlerContext ctx, Object evt) {
ctx.fireUserEventTriggered(evt);
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
logger.log(Level.SEVERE, cause.getMessage(), cause);
interaction.fail(ctx.channel(), cause);
ctx.close();
}
}

View file

@ -0,0 +1,255 @@
package org.xbib.net.http.client.netty.http1;
import io.netty.buffer.Unpooled;
import io.netty.channel.Channel;
import io.netty.handler.codec.http.DefaultFullHttpRequest;
import io.netty.handler.codec.http.FullHttpResponse;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http.HttpMethod;
import io.netty.handler.codec.http.HttpVersion;
import io.netty.handler.codec.http.multipart.DefaultHttpDataFactory;
import io.netty.handler.codec.http.multipart.HttpDataFactory;
import io.netty.handler.codec.http.multipart.HttpPostRequestEncoder;
import io.netty.handler.codec.http2.Http2Headers;
import io.netty.handler.codec.http2.Http2Settings;
import java.io.IOException;
import java.net.ConnectException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.xbib.net.URLSyntaxException;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpHeaders;
import org.xbib.net.http.HttpResponseStatus;
import org.xbib.net.http.cookie.Cookie;
import org.xbib.net.http.client.cookie.CookieDecoder;
import org.xbib.net.http.client.cookie.CookieEncoder;
import org.xbib.net.http.client.netty.BaseInteraction;
import org.xbib.net.http.client.netty.HttpResponseBuilder;
import org.xbib.net.http.client.netty.StreamIds;
import org.xbib.net.http.client.netty.http2.Http2Interaction;
import org.xbib.net.http.client.netty.HttpRequest;
import org.xbib.net.http.client.netty.HttpResponse;
import org.xbib.net.http.client.netty.Interaction;
import org.xbib.net.http.client.netty.NettyHttpClient;
public class Http1Interaction extends BaseInteraction {
private static final Logger logger = Logger.getLogger(Http1Interaction.class.getName());
private final HttpDataFactory httpDataFactory;
public Http1Interaction(NettyHttpClient nettyHttpClient, HttpAddress httpAddress) {
super(nettyHttpClient, httpAddress);
this.httpDataFactory = new DefaultHttpDataFactory();
}
@Override
public Interaction execute(HttpRequest request) throws IOException {
if (throwable != null) {
logger.log(Level.WARNING, throwable.getMessage(), throwable);
return this;
}
httpRequest = request;
Channel channel = acquireChannel(request);
try {
// if http2Settings is present, we have a HTTP-2 upgrade
waitForSettings(5L, TimeUnit.SECONDS);
if (http2Settings != null) {
Http2Interaction interaction = upgradeInteraction();
interaction.executeRequest(request, channel);
return interaction;
}
} catch (ExecutionException | InterruptedException | TimeoutException e) {
throw new IOException(e);
}
return executeRequest(request, channel);
}
public Interaction executeRequest(HttpRequest request, Channel channel) throws IOException {
final String channelId = channel.id().toString();
streamIds.putIfAbsent(channelId, new StreamIds());
// Some HTTP 1 servers do not understand URIs in HTTP command line in spite of RFC 7230.
// The "origin form" requires a "Host" header.
// Our algorithm is: use always "origin form" for HTTP 1, use absolute form for HTTP 2.
// The reason is that Netty derives the HTTP/2 scheme header from the absolute form.
String uri = request.getVersion().majorVersion() == 1 ? request.getURL().relativeReference() : request.getURL().toExternalForm();
HttpVersion httpVersion = HttpVersion.valueOf(request.getVersion().text());
HttpMethod httpMethod = HttpMethod.valueOf(request.getMethod().name());
DefaultFullHttpRequest fullHttpRequest = request.getBody() == null ?
new DefaultFullHttpRequest(httpVersion, httpMethod, uri) :
new DefaultFullHttpRequest(httpVersion, httpMethod, uri, Unpooled.wrappedBuffer(request.getBody()));
HttpPostRequestEncoder httpPostRequestEncoder = null;
final Integer streamId = streamIds.get(channelId).nextStreamId();
if (streamId == null) {
throw new IllegalStateException("stream id is null");
}
// add matching cookies from box (previous requests) and new cookies from request builder
Collection<Cookie> cookies = new ArrayList<>();
cookies.addAll(matchCookiesFromBox(request));
cookies.addAll(matchCookies(request));
if (!cookies.isEmpty()) {
request.getHeaders().set(HttpHeaderNames.COOKIE, CookieEncoder.STRICT.encode(cookies));
}
request.getHeaders().entries().forEach(p -> fullHttpRequest.headers().add(p.getKey(), p.getValue()));
if (request.getBody() == null && !request.getBodyData().isEmpty()) {
try {
httpPostRequestEncoder = new HttpPostRequestEncoder(httpDataFactory, fullHttpRequest, true);
httpPostRequestEncoder.setBodyHttpDatas(request.getBodyData());
httpPostRequestEncoder.finalizeRequest();
} catch (HttpPostRequestEncoder.ErrorDataEncoderException e) {
throw new IOException(e);
}
}
if (!channel.isWritable()) {
logger.log(Level.WARNING, "channel not writable");
return this;
}
channel.write(fullHttpRequest);
if (httpPostRequestEncoder != null && httpPostRequestEncoder.isChunked()) {
channel.write(httpPostRequestEncoder);
}
channel.flush();
if (httpPostRequestEncoder != null) {
httpPostRequestEncoder.cleanFiles();
}
return this;
}
@Override
public void settingsPrefaceWritten() {
logger.log(Level.FINEST, "settings/preface written");
}
@Override
public void waitForSettings(long value, TimeUnit timeUnit) throws ExecutionException, InterruptedException, TimeoutException {
if (settingsPromise != null) {
logger.log(Level.FINEST, "waiting for settings, promise = " + settingsPromise);
settingsPromise.get(value, timeUnit);
}
}
@Override
public void settingsReceived(Http2Settings http2Settings) {
this.http2Settings = http2Settings;
if (settingsPromise != null) {
logger.log(Level.FINEST, "received settings " + http2Settings + " for promise " + settingsPromise);
if (!settingsPromise.isDone()) {
settingsPromise.setSuccess();
}
} else {
logger.log(Level.WARNING, "settings received but no promise present");
}
}
@Override
public void responseReceived(Channel channel, Integer streamId, FullHttpResponse fullHttpResponse) {
if (throwable != null) {
logger.log(Level.WARNING, "throwable not null", throwable);
return;
}
HttpResponse httpResponse = null;
try {
// streamID is expected to be null, last request on memory
// is expected to be current, remove request from memory
for (String cookieString : fullHttpResponse.headers().getAll(HttpHeaderNames.SET_COOKIE)) {
Cookie cookie = CookieDecoder.STRICT.decode(cookieString);
addCookie(cookie);
}
HttpResponseStatus httpStatus = HttpResponseStatus.valueOf(fullHttpResponse.status().code());
HttpHeaders httpHeaders = new HttpHeaders();
fullHttpResponse.headers().iteratorCharSequence().forEachRemaining(e -> httpHeaders.add(e.getKey(), e.getValue().toString()));
httpResponse = newHttpResponseBuilder(channel)
.setHttpAddress(httpAddress)
.setLocalAddress(channel.localAddress())
.setRemoteAddress(channel.remoteAddress())
.setCookieBox(getCookieBox())
.setStatus(httpStatus)
.setHeaders(httpHeaders)
.setByteBuffer(fullHttpResponse.content().nioBuffer())
.build();
httpRequest.onResponse(httpResponse);
// check for retry / continue
try {
HttpRequest retryRequest = retry(httpRequest, httpResponse);
if (retryRequest != null) {
// retry transport, wait for completion
nettyHttpClient.retry(this, retryRequest);
} else {
HttpRequest continueRequest = continuation(httpRequest, httpResponse);
if (continueRequest != null) {
// continue with new transport, synchronous call here,
// wait for completion
nettyHttpClient.continuation(this, continueRequest);
}
}
} catch (URLSyntaxException | IOException e) {
logger.log(Level.WARNING, e.getMessage(), e);
}
// acknowledge success, if possible
String channelId = channel.id().toString();
StreamIds streamIds = super.streamIds.get(channelId);
if (streamIds != null) {
Integer lastKey = streamIds.lastKey();
if (lastKey != null) {
CompletableFuture<Boolean> promise = streamIds.get(lastKey);
if (promise != null) {
promise.complete(true);
}
}
}
} finally {
if (httpResponse != null) {
httpResponse.release();
}
}
}
@Override
public void pushPromiseReceived(Channel channel, Integer streamId,
Integer promisedStreamId, Http2Headers headers) {
}
@Override
protected String getRequestKey(String channelId, Integer streamId) {
return null;
}
@Override
protected Channel nextChannel() throws IOException {
Channel channel = newChannel(httpAddress);
if (channel == null) {
ConnectException connectException;
if (httpAddress != null) {
connectException = new ConnectException("unable to connect to " + httpAddress);
} else if (nettyHttpClient.hasPooledNodes()) {
connectException = new ConnectException("unable to get channel from pool");
} else {
// API misuse
connectException = new ConnectException("unable to get channel");
}
this.throwable = connectException;
throw connectException;
}
return channel;
}
@Override
public void close() throws IOException {
httpDataFactory.cleanAllHttpData();
super.close();
}
protected HttpResponseBuilder newHttpResponseBuilder(Channel channel) {
return HttpResponse.builder();
}
protected Http2Interaction upgradeInteraction() {
return new Http2Interaction(nettyHttpClient, httpAddress);
}
}

View file

@ -0,0 +1,84 @@
package org.xbib.net.http.client.netty.http2;
import io.netty.channel.Channel;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.handler.codec.http2.Http2FrameLogger;
import io.netty.handler.codec.http2.Http2MultiplexCodec;
import io.netty.handler.codec.http2.Http2MultiplexCodecBuilder;
import io.netty.handler.logging.LogLevel;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpVersion;
import org.xbib.net.http.client.netty.HttpChannelInitializer;
import org.xbib.net.http.client.netty.Interaction;
import org.xbib.net.http.client.netty.NettyCustomizer;
import org.xbib.net.http.client.netty.NettyHttpClient;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.TrafficLoggingHandler;
public class Http2ChannelInitializer implements HttpChannelInitializer {
private static final Logger logger = Logger.getLogger(Http2ChannelInitializer.class.getName());
public Http2ChannelInitializer() {
}
@Override
public boolean supports(HttpAddress httpAddress) {
return HttpVersion.HTTP_2_0.equals(httpAddress.getVersion()) && !httpAddress.isSecure();
}
@Override
public Interaction newInteraction(NettyHttpClient client, HttpAddress httpAddress) {
return new Http2Interaction(client, httpAddress);
}
@Override
public void init(Channel channel,
HttpAddress httpAddress,
NettyHttpClient nettyHttpClient,
NettyCustomizer nettyCustomizer,
Interaction interaction) {
NettyHttpClientConfig nettyHttpClientConfig = nettyHttpClient.getClientConfig();
ChannelPipeline pipeline = channel.pipeline();
if (nettyHttpClientConfig.isDebug()) {
pipeline.addLast(new TrafficLoggingHandler(LogLevel.DEBUG));
}
configurePlain(channel, nettyHttpClient, interaction);
if (nettyCustomizer != null) {
nettyCustomizer.afterChannelInitialized(channel);
}
if (nettyHttpClientConfig.isDebug()) {
logger.log(Level.FINE, "HTTP/2 plain channel initialized: address = " + httpAddress +
" pipeline = " + pipeline.names());
}
}
private void configurePlain(Channel channel,
NettyHttpClient nettyHttpClient,
Interaction interaction) {
NettyHttpClientConfig nettyHttpClientConfig = nettyHttpClient.getClientConfig();
ChannelInitializer<Channel> initializer = new ChannelInitializer<>() {
@Override
protected void initChannel(Channel ch) {
throw new IllegalStateException();
}
};
Http2MultiplexCodecBuilder multiplexCodecBuilder = Http2MultiplexCodecBuilder.forClient(initializer)
.initialSettings(nettyHttpClientConfig.getHttp2Settings());
if (nettyHttpClientConfig.isDebug()) {
multiplexCodecBuilder.frameLogger(new Http2FrameLogger(LogLevel.DEBUG, "client-frame"));
}
Http2MultiplexCodec multiplexCodec = multiplexCodecBuilder
.autoAckPingFrame(true)
.autoAckSettingsFrame(true)
.decoupleCloseAndGoAway(false)
.gracefulShutdownTimeoutMillis(30000L)
.build();
ChannelPipeline pipeline = channel.pipeline();
pipeline.addLast("client-multiplex", multiplexCodec);
pipeline.addLast("client-messages", new Http2Messages(interaction));
}
}

View file

@ -0,0 +1,38 @@
package org.xbib.net.http.client.netty.http2;
import io.netty.channel.Channel;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.handler.codec.http.HttpContentDecompressor;
import io.netty.handler.codec.http.HttpObjectAggregator;
import io.netty.handler.codec.http2.Http2StreamFrameToHttpObjectCodec;
import org.xbib.net.http.client.netty.Interaction;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
public class Http2ChildChannelInitializer extends ChannelInitializer<Channel> {
private final NettyHttpClientConfig clientConfig;
private final Interaction interaction;
protected final Channel parentChannel;
public Http2ChildChannelInitializer(NettyHttpClientConfig clientConfig, Interaction interaction, Channel parentChannel) {
this.clientConfig = clientConfig;
this.interaction = interaction;
this.parentChannel = parentChannel;
}
@Override
protected void initChannel(Channel ch) {
ChannelPipeline p = ch.pipeline();
p.addLast("child-client-frame-converter",
new Http2StreamFrameToHttpObjectCodec(false));
p.addLast("child-client-decompressor",
new HttpContentDecompressor());
p.addLast("child-client-chunk-aggregator",
new HttpObjectAggregator(clientConfig.getMaxContentLength()));
p.addLast("child-client-response-handler",
new Http2Handler(interaction));
}
}

View file

@ -0,0 +1,50 @@
package org.xbib.net.http.client.netty.http2;
import io.netty.channel.ChannelDuplexHandler;
import io.netty.channel.ChannelHandler;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.http.FullHttpResponse;
import io.netty.handler.codec.http2.HttpConversionUtil;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.xbib.net.http.client.netty.Interaction;
@ChannelHandler.Sharable
public class Http2Handler extends ChannelDuplexHandler {
private static final Logger logger = Logger.getLogger(Http2Handler.class.getName());
private final Interaction interaction;
public Http2Handler(Interaction interaction) {
this.interaction = interaction;
}
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
if (msg instanceof FullHttpResponse) {
FullHttpResponse httpResponse = (FullHttpResponse) msg;
try {
Integer streamId = httpResponse.headers().getInt(HttpConversionUtil.ExtensionHeaderNames.STREAM_ID.text());
interaction.responseReceived(ctx.channel(), streamId, httpResponse);
} finally {
httpResponse.release();
}
} else {
super.channelRead(ctx, msg);
}
}
@Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
ctx.fireChannelInactive();
interaction.inactive(ctx.channel());
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
logger.log(Level.FINE, "exception caught");
interaction.fail(ctx.channel(), cause);
ctx.close();
}
}

View file

@ -0,0 +1,254 @@
package org.xbib.net.http.client.netty.http2;
import io.netty.buffer.Unpooled;
import io.netty.channel.Channel;
import io.netty.channel.ChannelInitializer;
import io.netty.handler.codec.http.DefaultHttpHeaders;
import io.netty.handler.codec.http.FullHttpResponse;
import io.netty.handler.codec.http.HttpHeaderNames;
import io.netty.handler.codec.http2.DefaultHttp2DataFrame;
import io.netty.handler.codec.http2.DefaultHttp2Headers;
import io.netty.handler.codec.http2.DefaultHttp2HeadersFrame;
import io.netty.handler.codec.http2.Http2Headers;
import io.netty.handler.codec.http2.Http2Settings;
import io.netty.handler.codec.http2.Http2StreamChannel;
import io.netty.handler.codec.http2.Http2StreamChannelBootstrap;
import io.netty.handler.codec.http2.HttpConversionUtil;
import java.io.IOException;
import java.net.ConnectException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.xbib.net.URLSyntaxException;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpHeaders;
import org.xbib.net.http.HttpResponseStatus;
import org.xbib.net.http.cookie.Cookie;
import org.xbib.net.http.client.cookie.CookieDecoder;
import org.xbib.net.http.client.cookie.CookieEncoder;
import org.xbib.net.http.client.netty.BaseInteraction;
import org.xbib.net.http.client.netty.HttpResponseBuilder;
import org.xbib.net.http.client.netty.NettyHttpClientConfig;
import org.xbib.net.http.client.netty.StreamIds;
import org.xbib.net.http.client.netty.HttpRequest;
import org.xbib.net.http.client.netty.HttpResponse;
import org.xbib.net.http.client.netty.Interaction;
import org.xbib.net.http.client.netty.NettyHttpClient;
public class Http2Interaction extends BaseInteraction {
private static final Logger logger = Logger.getLogger(Http2Interaction.class.getName());
public Http2Interaction(NettyHttpClient nettyHttpClient, HttpAddress httpAddress) {
super(nettyHttpClient, httpAddress);
}
@Override
public Interaction execute(HttpRequest request) throws IOException {
if (throwable != null) {
return this;
}
Channel channel = acquireChannel(request);
try {
waitForSettings(5L, TimeUnit.SECONDS);
} catch (ExecutionException | InterruptedException | TimeoutException e) {
throw new IOException(e);
}
return executeRequest(request, channel);
}
public Interaction executeRequest(HttpRequest request, Channel channel) throws IOException {
this.httpRequest = request;
final String channelId = channel.id().toString();
streamIds.putIfAbsent(channelId, new StreamIds());
ChannelInitializer<Channel> initializer = newHttp2ChildChannelInitializer(nettyHttpClient.getClientConfig(), this, channel);
Http2StreamChannel childChannel = new Http2StreamChannelBootstrap(channel)
.handler(initializer).open().syncUninterruptibly().getNow();
CharSequence method = request.getMethod().name();
String scheme = request.getURL().getScheme();
String authority = request.getURL().getHost() + (request.getURL().getPort() != null ? ":" + request.getURL().getPort() : "");
String relative = request.getURL().relativeReference();
String path = relative.isEmpty() ? "/" : relative;
Http2Headers http2Headers = new DefaultHttp2Headers()
.method(method).scheme(scheme).authority(authority).path(path);
StreamIds streamIds = super.streamIds.get(channelId);
if (streamIds == null) {
throw new IllegalStateException();
}
final Integer streamId = streamIds.nextStreamId();
if (streamId == null) {
throw new IllegalStateException();
}
http2Headers.setInt(HttpConversionUtil.ExtensionHeaderNames.STREAM_ID.text(), streamId);
// add matching cookies from box (previous requests) and new cookies from request builder
Collection<Cookie> cookies = new ArrayList<>();
cookies.addAll(matchCookiesFromBox(request));
cookies.addAll(matchCookies(request));
if (!cookies.isEmpty()) {
request.getHeaders().set(HttpHeaderNames.COOKIE, CookieEncoder.STRICT.encode(cookies));
}
DefaultHttpHeaders httpHeaders = new DefaultHttpHeaders();
request.getHeaders().entries().forEach(p -> httpHeaders.set(p.getKey(), p.getValue()));
HttpConversionUtil.toHttp2Headers(httpHeaders, http2Headers);
boolean hasContent = request.getBody() != null && request.getBody().remaining() > 0;
DefaultHttp2HeadersFrame headersFrame = new DefaultHttp2HeadersFrame(http2Headers, !hasContent);
DefaultHttp2DataFrame dataFrame;
childChannel.write(headersFrame);
if (hasContent) {
dataFrame = new DefaultHttp2DataFrame(Unpooled.wrappedBuffer(request.getBody()), true);
childChannel.write(dataFrame);
}
childChannel.flush();
if (nettyHttpClient.hasPooledNodes()) {
releaseChannel(channel, false);
}
return this;
}
@Override
public void settingsPrefaceWritten() {
logger.log(Level.FINEST, "settings/preface written");
}
@Override
public void waitForSettings(long value, TimeUnit timeUnit) throws ExecutionException, InterruptedException, TimeoutException {
if (settingsPromise != null) {
logger.log(Level.FINEST, "waiting for settings, promise = " + settingsPromise);
settingsPromise.get(value, timeUnit);
}
}
@Override
public void settingsReceived(Http2Settings http2Settings) {
this.http2Settings = http2Settings;
if (settingsPromise != null) {
logger.log(Level.FINEST, "received settings for promise = " + settingsPromise);
settingsPromise.setSuccess();
} else {
logger.log(Level.WARNING, "settings received but no promise present");
}
}
@Override
public void responseReceived(Channel channel, Integer streamId, FullHttpResponse fullHttpResponse) {
if (throwable != null) {
logger.log(Level.WARNING, "throwable is not null?", throwable);
return;
}
if (streamId == null) {
logger.log(Level.WARNING, "stream ID is null?");
return;
}
HttpResponse httpResponse = null;
try {
// format of childchan channel ID is <parent channel ID> "/" <substream ID>
String channelId = channel.id().toString();
int pos = channelId.indexOf('/');
channelId = pos > 0 ? channelId.substring(0, pos) : channelId;
StreamIds streamIds = super.streamIds.get(channelId);
if (streamIds == null) {
// should never happen
if (logger.isLoggable(Level.WARNING)) {
logger.log(Level.WARNING, "stream ID is null? channelId = " + channelId);
}
return;
}
for (String cookieString : fullHttpResponse.headers().getAll(HttpHeaderNames.SET_COOKIE)) {
Cookie cookie = CookieDecoder.STRICT.decode(cookieString);
addCookie(cookie);
}
HttpResponseStatus httpStatus = HttpResponseStatus.valueOf(fullHttpResponse.status().code());
HttpHeaders httpHeaders = new HttpHeaders();
fullHttpResponse.headers().iteratorCharSequence().forEachRemaining(e -> httpHeaders.add(e.getKey(), e.getValue().toString()));
httpResponse = newHttpResponseBuilder(channel)
.setHttpAddress(httpAddress)
.setCookieBox(getCookieBox())
.setStatus(httpStatus)
.setHeaders(httpHeaders)
.setByteBuffer(fullHttpResponse.content().nioBuffer())
.build();
CompletableFuture<Boolean> promise = streamIds.get(streamId);
try {
httpRequest.onResponse(httpResponse);
HttpRequest retryRequest = retry(httpRequest, httpResponse);
if (retryRequest != null) {
// retry transport, wait for completion
nettyHttpClient.retry(this, retryRequest);
} else {
HttpRequest continueRequest = continuation(httpRequest, httpResponse);
if (continueRequest != null) {
// continue with new transport, synchronous call here, wait for completion
nettyHttpClient.continuation(this, continueRequest);
}
}
if (promise != null) {
promise.complete(true);
} else {
// when transport is closed, stream IDs will be emptied
logger.log(Level.FINE, "promise is null, streamIDs lost");
}
} catch (URLSyntaxException | IOException e) {
if (promise != null) {
promise.completeExceptionally(e);
} else {
logger.log(Level.FINE, "promise is null, can't abort");
}
} finally {
streamIds.remove(streamId);
}
} finally {
if (httpResponse != null) {
httpResponse.release();
}
}
}
@Override
public void pushPromiseReceived(Channel channel, Integer streamId, Integer promisedStreamId, Http2Headers headers) {
String channelId = channel.id().toString();
StreamIds streamIds = super.streamIds.get(channelId);
if (streamIds != null) {
streamIds.put(promisedStreamId, new CompletableFuture<>());
}
}
@Override
protected String getRequestKey(String channelId, Integer streamId) {
return channelId + "#" + streamId;
}
@Override
protected Channel nextChannel() throws IOException {
Channel channel = newChannel(httpAddress);
if (channel == null) {
ConnectException connectException;
if (httpAddress != null) {
connectException = new ConnectException("unable to connect to " + httpAddress);
} else if (nettyHttpClient.hasPooledNodes()) {
connectException = new ConnectException("unable to get channel from pool");
} else {
// API misuse
connectException = new ConnectException("unable to get channel");
}
this.throwable = connectException;
throw connectException;
}
return channel;
}
protected Http2ChildChannelInitializer newHttp2ChildChannelInitializer(NettyHttpClientConfig clientConfig,
Http2Interaction interaction,
Channel parentChannel) {
return new Http2ChildChannelInitializer(clientConfig, interaction, parentChannel);
}
protected HttpResponseBuilder newHttpResponseBuilder(Channel channel) {
return HttpResponse.builder();
}
}

View file

@ -0,0 +1,44 @@
package org.xbib.net.http.client.netty.http2;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.handler.codec.http2.DefaultHttp2SettingsFrame;
import io.netty.handler.codec.http2.Http2ConnectionPrefaceAndSettingsFrameWrittenEvent;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.xbib.net.http.client.netty.Interaction;
public class Http2Messages extends ChannelInboundHandlerAdapter {
private static final Logger logger = Logger.getLogger(Http2Messages.class.getName());
private final Interaction interaction;
public Http2Messages(Interaction interaction) {
this.interaction = interaction;
}
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
if (msg instanceof DefaultHttp2SettingsFrame) {
DefaultHttp2SettingsFrame settingsFrame = (DefaultHttp2SettingsFrame) msg;
interaction.settingsReceived(settingsFrame.settings());
logger.log(Level.FINEST, "received settings ");
}
}
@Override
public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception {
if (evt instanceof Http2ConnectionPrefaceAndSettingsFrameWrittenEvent) {
Http2ConnectionPrefaceAndSettingsFrameWrittenEvent event =
(Http2ConnectionPrefaceAndSettingsFrameWrittenEvent) evt;
logger.log(Level.FINEST, "received preface and setting written event " + event);
}
ctx.fireUserEventTriggered(evt);
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
interaction.fail(ctx.channel(), cause);
}
}

View file

@ -0,0 +1 @@
org.xbib.net.http.client.netty.NioClientTransportProvider

View file

@ -0,0 +1,2 @@
org.xbib.net.http.client.netty.http1.Http1ChannelInitializer
org.xbib.net.http.client.netty.http2.Http2ChannelInitializer

View file

@ -0,0 +1,37 @@
package org.xbib.net.http.client.netty;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertTrue;
class Http1Test {
private static final Logger logger = Logger.getLogger(Http1Test.class.getName());
@Test
void testHttpGetRequest() throws Exception {
NettyHttpClientConfig config = new NettyHttpClientConfig()
.setDebug(true);
AtomicBoolean received = new AtomicBoolean();
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("http://httpbin.org")
.setResponseListener(resp -> {
logger.log(Level.INFO,
"local address = " + resp.getLocalAddress() +
" got response = " + resp.getHeaders() +
resp.getBodyAsChars(StandardCharsets.UTF_8) +
" status=" + resp.getStatus());
received.set(true);
})
.build();
client.execute(request).get().close();
}
assertTrue(received.get());
}
}

View file

@ -0,0 +1,43 @@
package org.xbib.net.http.client.netty;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.xbib.net.http.HttpVersion;
class Http2Test {
private static final Logger logger = Logger.getLogger(Http2Test.class.getName());
/**
* HTTP/2 cleartext is not support by many servers.
* This will return HTTP/1.1 Bad request and we run into a timeout.
*/
@Test
void testCleartext() {
Assertions.assertThrows(IOException.class, () -> {
NettyHttpClientConfig config = new NettyHttpClientConfig()
.setDebug(true);
try (NettyHttpClient client = NettyHttpClient.builder()
.setConfig(config)
.build()) {
HttpRequest request = HttpRequest.get()
.setURL("http://httpbin.org")
.setVersion(HttpVersion.HTTP_2_0)
.setResponseListener(resp -> {
logger.log(Level.INFO,
"local address = " + resp.getLocalAddress() +
" got respons =: " + resp.getHeaders() +
resp.getBodyAsChars(StandardCharsets.UTF_8) +
" status=" + resp.getStatus());
})
.build();
client.execute(request).get().close();
}
});
}
}

View file

@ -0,0 +1,5 @@
handlers=java.util.logging.ConsoleHandler
.level=ALL
java.util.logging.ConsoleHandler.level=ALL
java.util.logging.ConsoleHandler.formatter=org.xbib.net.util.ThreadLoggingFormatter
jdk.event.security.level=INFO

View file

@ -0,0 +1,3 @@
dependencies {
api project(':net-http-client')
}

View file

@ -0,0 +1,6 @@
module org.xbib.net.http.client.jdk {
requires org.xbib.net;
requires org.xbib.net.http;
requires org.xbib.net.http.client;
requires java.logging;
}

View file

@ -0,0 +1,28 @@
package org.xbib.net.http.client.jdk;
import org.xbib.net.http.client.HttpClient;
import org.xbib.net.http.client.HttpRequest;
import org.xbib.net.http.client.HttpResponse;
import java.io.IOException;
import java.util.concurrent.CompletableFuture;
import java.util.function.Function;
public class JdkHttpClient implements HttpClient<HttpRequest, HttpResponse> {
private final JdkHttpClientBuilder builder;
JdkHttpClient(JdkHttpClientBuilder builder) {
this.builder = builder;
}
@Override
public <T> CompletableFuture<T> execute(HttpRequest request, Function<HttpResponse, T> supplier) throws IOException {
return null;
}
@Override
public void close() throws IOException {
}
}

View file

@ -0,0 +1,39 @@
package org.xbib.net.http.client.jdk;
import java.io.IOException;
import java.security.Provider;
import java.security.Security;
import java.util.Arrays;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.stream.Collectors;
public class JdkHttpClientBuilder {
private static final Logger logger = Logger.getLogger(JdkHttpClientBuilder.class.getName());
JdkHttpClientConfig jdkHttpClientConfig;
JdkHttpClientBuilder() {
}
public JdkHttpClientBuilder setConfig(JdkHttpClientConfig JdkHttpClientConfig) {
this.jdkHttpClientConfig = JdkHttpClientConfig;
return this;
}
public JdkHttpClient build() throws IOException {
if (logger.isLoggable(Level.FINEST)) {
logger.log(Level.FINEST, "installed security providers = " +
Arrays.stream(Security.getProviders()).map(Provider::getName).collect(Collectors.toList()));
}
if (jdkHttpClientConfig == null) {
jdkHttpClientConfig = createEmptyConfig();
}
return new JdkHttpClient(this);
}
protected JdkHttpClientConfig createEmptyConfig() {
return new JdkHttpClientConfig();
}
}

View file

@ -0,0 +1,183 @@
package org.xbib.net.http.client.jdk;
import org.xbib.net.SocketConfig;
import org.xbib.net.http.client.BackOff;
import java.util.logging.Level;
public class JdkHttpClientConfig {
/**
* If frame logging /traffic logging is enabled or not.
*/
private boolean debug = false;
/**
* Default debug log level.
*/
private Level debugLogLevel = Level.FINE;
SocketConfig socketConfig = new SocketConfig();
private String transportProviderName = null;
/**
* If set to 0, then Netty will decide about thread count.
* Default is Runtime.getRuntime().availableProcessors() * 2
*/
private int threadCount = 0;
/**
* Set HTTP initial line length to 4k.
*/
private int maxInitialLineLength = 4 * 1024;
/**
* Set HTTP maximum headers size to 8k.
*/
private int maxHeadersSize = 8 * 1024;
/**
* Set HTTP chunk maximum size to 8k.
*/
private int maxChunkSize = 8 * 1024;
/**
* Set maximum content length to 256 MB.
*/
private int maxContentLength = 256 * 1024 * 1024;
/**
* This is Netty's default.
*/
private int maxCompositeBufferComponents = 1024;
/**
* Default for gzip codec is true
*/
private boolean gzipEnabled = false;
private BackOff backOff = BackOff.ZERO_BACKOFF;
public JdkHttpClientConfig() {
}
public JdkHttpClientConfig setDebug(boolean debug) {
this.debug = debug;
return this;
}
public JdkHttpClientConfig enableDebug() {
this.debug = true;
return this;
}
public JdkHttpClientConfig disableDebug() {
this.debug = false;
return this;
}
public boolean isDebug() {
return debug;
}
public JdkHttpClientConfig setDebugLogLevel(Level debugLogLevel) {
this.debugLogLevel = debugLogLevel;
return this;
}
public Level getDebugLogLevel() {
return debugLogLevel;
}
public JdkHttpClientConfig setTransportProviderName(String transportProviderName) {
this.transportProviderName = transportProviderName;
return this;
}
public String getTransportProviderName() {
return transportProviderName;
}
public JdkHttpClientConfig setThreadCount(int threadCount) {
this.threadCount = threadCount;
return this;
}
public int getThreadCount() {
return threadCount;
}
public JdkHttpClientConfig setSocketConfig(SocketConfig socketConfig) {
this.socketConfig = socketConfig;
return this;
}
public SocketConfig getSocketConfig() {
return socketConfig;
}
public JdkHttpClientConfig setMaxInitialLineLength(int maxInitialLineLength) {
this.maxInitialLineLength = maxInitialLineLength;
return this;
}
public int getMaxInitialLineLength() {
return maxInitialLineLength;
}
public JdkHttpClientConfig setMaxHeadersSize(int maxHeadersSize) {
this.maxHeadersSize = maxHeadersSize;
return this;
}
public int getMaxHeadersSize() {
return maxHeadersSize;
}
public JdkHttpClientConfig setMaxChunkSize(int maxChunkSize) {
this.maxChunkSize = maxChunkSize;
return this;
}
public int getMaxChunkSize() {
return maxChunkSize;
}
public JdkHttpClientConfig setMaxContentLength(int maxContentLength) {
this.maxContentLength = maxContentLength;
return this;
}
public int getMaxContentLength() {
return maxContentLength;
}
public JdkHttpClientConfig setMaxCompositeBufferComponents(int maxCompositeBufferComponents) {
this.maxCompositeBufferComponents = maxCompositeBufferComponents;
return this;
}
public int getMaxCompositeBufferComponents() {
return maxCompositeBufferComponents;
}
public JdkHttpClientConfig setGzipEnabled(boolean gzipEnabled) {
this.gzipEnabled = gzipEnabled;
return this;
}
public boolean isGzipEnabled() {
return gzipEnabled;
}
public JdkHttpClientConfig setBackOff(BackOff backOff) {
this.backOff = backOff;
return this;
}
public BackOff getBackOff() {
return backOff;
}
}

View file

@ -0,0 +1,3 @@
dependencies {
api project(':net-http')
}

View file

@ -0,0 +1,7 @@
module org.xbib.net.http.client {
exports org.xbib.net.http.client;
exports org.xbib.net.http.client.cookie;
requires org.xbib.net;
requires org.xbib.net.http;
requires java.logging;
}

View file

@ -0,0 +1,65 @@
package org.xbib.net.http.client;
/**
* Back-off policy when retrying an operation.
*/
public interface BackOff {
/**
* Indicates that no more retries should be made for use in {@link #nextBackOffMillis()}. */
long STOP = -1L;
/**
* Reset to initial state.
*/
void reset();
/**
* Gets the number of milliseconds to wait before retrying the operation or {@link #STOP} to
* indicate that no retries should be made.
*
* @return milliseconds before operation retry
*
* <p>
* Example usage:
* </p>
*
* <pre>
long backOffMillis = backoff.nextBackOffMillis();
if (backOffMillis == Backoff.STOP) {
// do not retry operation
} else {
// sleep for backOffMillis milliseconds and retry operation
}
* </pre>
*/
long nextBackOffMillis();
/**
* Fixed back-off policy whose back-off time is always zero, meaning that the operation is retried
* immediately without waiting.
*/
BackOff ZERO_BACKOFF = new BackOff() {
public void reset() {
}
public long nextBackOffMillis() {
return 0;
}
};
/**
* Fixed back-off policy that always returns {@code #STOP} for {@link #nextBackOffMillis()},
* meaning that the operation should not be retried.
*/
BackOff STOP_BACKOFF = new BackOff() {
public void reset() {
}
public long nextBackOffMillis() {
return STOP;
}
};
}

View file

@ -0,0 +1,65 @@
package org.xbib.net.http.client;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.Charset;
import org.xbib.net.ParameterBuilder;
import org.xbib.net.URL;
import org.xbib.net.http.HttpHeaders;
import org.xbib.net.http.HttpMethod;
import org.xbib.net.http.HttpVersion;
public abstract class BaseHttpRequest implements HttpRequest {
protected final BaseHttpRequestBuilder builder;
protected BaseHttpRequest(BaseHttpRequestBuilder builder) {
this.builder = builder;
}
@Override
public InetSocketAddress getLocalAddress() {
return builder.localAddress;
}
@Override
public InetSocketAddress getRemoteAddress() {
return builder.remoteAddress;
}
@Override
public URL getBaseURL() {
return builder.url;
}
@Override
public HttpVersion getVersion() {
return builder.httpVersion;
}
@Override
public HttpMethod getMethod() {
return builder.httpMethod;
}
@Override
public HttpHeaders getHeaders() {
return builder.httpHeaders;
}
@Override
public ParameterBuilder getParameters() {
return builder.parameterBuilder;
}
@Override
public ByteBuffer getBody() {
return builder.byteBuffer;
}
@Override
public CharBuffer getBodyAsChars(Charset charset) {
return builder.byteBuffer != null ? charset.decode(builder.byteBuffer) : null;
}
}

View file

@ -0,0 +1,124 @@
package org.xbib.net.http.client;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import org.xbib.net.ParameterBuilder;
import org.xbib.net.URL;
import org.xbib.net.http.HttpAddress;
import org.xbib.net.http.HttpHeaders;
import org.xbib.net.http.HttpMethod;
import org.xbib.net.http.HttpVersion;
public abstract class BaseHttpRequestBuilder implements HttpRequestBuilder {
HttpAddress httpAddress;
InetSocketAddress localAddress;
InetSocketAddress remoteAddress;
URL url;
String requestPath;
ParameterBuilder parameterBuilder;
Integer sequenceId;
Integer streamId;
Long requestId;
HttpVersion httpVersion;
HttpMethod httpMethod;
HttpHeaders httpHeaders = new HttpHeaders();
ByteBuffer byteBuffer;
protected BaseHttpRequestBuilder() {
}
public BaseHttpRequestBuilder setVersion(HttpVersion httpVersion) {
this.httpVersion = httpVersion;
return this;
}
public HttpVersion getVersion() {
return httpVersion;
}
public BaseHttpRequestBuilder setMethod(HttpMethod httpMethod) {
this.httpMethod = httpMethod;
return this;
}
public HttpMethod getMethod() {
return httpMethod;
}
public BaseHttpRequestBuilder setHeaders(HttpHeaders httpHeaders) {
this.httpHeaders = httpHeaders;
return this;
}
public BaseHttpRequestBuilder addHeader(String key, String value) {
this.httpHeaders.add(key, value);
return this;
}
@Override
public BaseHttpRequestBuilder setAddress(HttpAddress httpAddress) {
this.httpAddress = httpAddress;
return this;
}
@Override
public BaseHttpRequestBuilder setURL(URL url) {
this.url = url;
return this;
}
@Override
public BaseHttpRequestBuilder setRequestPath(String requestPath) {
this.requestPath = requestPath;
return this;
}
@Override
public BaseHttpRequestBuilder setParameterBuilder(ParameterBuilder parameterBuilder) {
this.parameterBuilder = parameterBuilder;
return this;
}
@Override
public BaseHttpRequestBuilder setBody(ByteBuffer byteBuffer) {
this.byteBuffer = byteBuffer;
return this;
}
public BaseHttpRequestBuilder setLocalAddress(InetSocketAddress localAddress) {
this.localAddress = localAddress;
return this;
}
public BaseHttpRequestBuilder setRemoteAddress(InetSocketAddress remoteAddress) {
this.remoteAddress = remoteAddress;
return this;
}
public BaseHttpRequestBuilder setSequenceId(Integer sequenceId) {
this.sequenceId = sequenceId;
return this;
}
public BaseHttpRequestBuilder setStreamId(Integer streamId) {
this.streamId = streamId;
return this;
}
public BaseHttpRequestBuilder setRequestId(Long requestId) {
this.requestId = requestId;
return this;
}
}

View file

@ -0,0 +1,8 @@
package org.xbib.net.http.client;
/**
* Client authentication modes, useful for SSL channels.
*/
public enum ClientAuthMode {
NONE, WANT, NEED
}

View file

@ -0,0 +1,7 @@
package org.xbib.net.http.client;
@FunctionalInterface
public interface ExceptionListener {
void onException(Throwable throwable);
}

View file

@ -0,0 +1,489 @@
package org.xbib.net.http.client;
/**
* Implementation of {@link BackOff} that increases the back off period for each retry attempt using
* a randomization function that grows exponentially.
*
* <p>
* {@link #nextBackOffMillis()} is calculated using the following formula:
* </p>
*
* <pre>
randomized_interval =
retry_interval * (random value in range [1 - randomization_factor, 1 + randomization_factor])
* </pre>
*
* <p>
* In other words {@link #nextBackOffMillis()} will range between the randomization factor
* percentage below and above the retry interval. For example, using 2 seconds as the base retry
* interval and 0.5 as the randomization factor, the actual back off period used in the next retry
* attempt will be between 1 and 3 seconds.
* </p>
*
* <p>
* <b>Note:</b> max_interval caps the retry_interval and not the randomized_interval.
* </p>
*
* <p>
* If the time elapsed since an {@link ExponentialBackOff} instance is created goes past the
* max_elapsed_time then the method {@link #nextBackOffMillis()} starts returning
* {@link BackOff#STOP}. The elapsed time can be reset by calling {@link #reset()}.
* </p>
*
* <p>
* Example: The default retry_interval is .5 seconds, default randomization_factor is 0.5, default
* multiplier is 1.5 and the default max_interval is 1 minute. For 10 tries the sequence will be
* (values in seconds) and assuming we go over the max_elapsed_time on the 10th try:
* </p>
*
* <pre>
request# retry_interval randomized_interval
1 0.5 [0.25, 0.75]
2 0.75 [0.375, 1.125]
3 1.125 [0.562, 1.687]
4 1.687 [0.8435, 2.53]
5 2.53 [1.265, 3.795]
6 3.795 [1.897, 5.692]
7 5.692 [2.846, 8.538]
8 8.538 [4.269, 12.807]
9 12.807 [6.403, 19.210]
10 19.210 {@link BackOff#STOP}
* </pre>
*
* <p>
* Implementation is not thread-safe.
* </p>
*/
public class ExponentialBackOff implements BackOff {
/** The default initial interval value in milliseconds (0.5 seconds). */
public static final int DEFAULT_INITIAL_INTERVAL_MILLIS = 500;
/**
* The default randomization factor (0.5 which results in a random period ranging between 50%
* below and 50% above the retry interval).
*/
public static final double DEFAULT_RANDOMIZATION_FACTOR = 0.5;
/** The default multiplier value (1.5 which is 50% increase per back off). */
public static final double DEFAULT_MULTIPLIER = 1.5;
/** The default maximum back off time in milliseconds (1 minute). */
public static final int DEFAULT_MAX_INTERVAL_MILLIS = 60000;
/** The default maximum elapsed time in milliseconds (15 minutes). */
public static final int DEFAULT_MAX_ELAPSED_TIME_MILLIS = 900000;
/** The current retry interval in milliseconds. */
private int currentIntervalMillis;
/** The initial retry interval in milliseconds. */
private final int initialIntervalMillis;
/**
* The randomization factor to use for creating a range around the retry interval.
*
* <p>
* A randomization factor of 0.5 results in a random period ranging between 50% below and 50%
* above the retry interval.
* </p>
*/
private final double randomizationFactor;
/** The value to multiply the current interval with for each retry attempt. */
private final double multiplier;
/**
* The maximum value of the back off period in milliseconds. Once the retry interval reaches this
* value it stops increasing.
*/
private final int maxIntervalMillis;
/**
* The system time in nanoseconds. It is calculated when an ExponentialBackOffPolicy instance is
* created and is reset when {@link #reset()} is called.
*/
private long startTimeNanos;
/**
* The maximum elapsed time after instantiating {@link ExponentialBackOff} or calling
* {@link #reset()} after which {@link #nextBackOffMillis()} returns {@link BackOff#STOP}.
*/
private final int maxElapsedTimeMillis;
/** Nano clock. */
private final NanoClock nanoClock;
/**
* Creates an instance of ExponentialBackOffPolicy using default values.
*
* <p>
* To override the defaults use {@link Builder}.
* </p>
*
* <ul>
* <li>{@code initialIntervalMillis} defaults to {@link #DEFAULT_INITIAL_INTERVAL_MILLIS}</li>
* <li>{@code randomizationFactor} defaults to {@link #DEFAULT_RANDOMIZATION_FACTOR}</li>
* <li>{@code multiplier} defaults to {@link #DEFAULT_MULTIPLIER}</li>
* <li>{@code maxIntervalMillis} defaults to {@link #DEFAULT_MAX_INTERVAL_MILLIS}</li>
* <li>{@code maxElapsedTimeMillis} defaults in {@link #DEFAULT_MAX_ELAPSED_TIME_MILLIS}</li>
* </ul>
*/
public ExponentialBackOff() {
this(new Builder());
}
/**
* @param builder builder
*/
private ExponentialBackOff(Builder builder) {
initialIntervalMillis = builder.initialIntervalMillis;
randomizationFactor = builder.randomizationFactor;
multiplier = builder.multiplier;
maxIntervalMillis = builder.maxIntervalMillis;
maxElapsedTimeMillis = builder.maxElapsedTimeMillis;
nanoClock = builder.nanoClock;
reset();
}
/**
* Sets the interval back to the initial retry interval and restarts the timer.
*/
public final void reset() {
currentIntervalMillis = initialIntervalMillis;
startTimeNanos = nanoClock.nanoTime();
}
public void setStartTimeNanos(long startTimeNanos) {
this.startTimeNanos = startTimeNanos;
}
/**
* {@inheritDoc}
*
* <p>
* This method calculates the next back off interval using the formula: randomized_interval =
* retry_interval +/- (randomization_factor * retry_interval)
* </p>
*
* <p>
* Subclasses may override if a different algorithm is required.
* </p>
*/
public long nextBackOffMillis() {
// Make sure we have not gone over the maximum elapsed time.
if (getElapsedTimeMillis() > maxElapsedTimeMillis) {
return STOP;
}
int randomizedInterval =
getRandomValueFromInterval(randomizationFactor, Math.random(), currentIntervalMillis);
incrementCurrentInterval();
return randomizedInterval;
}
/**
* Returns a random value from the interval [randomizationFactor * currentInterval,
* randomizationFactor * currentInterval].
* @param randomizationFactor the randomization factor
* @param random scaling factor
* @param currentIntervalMillis milliseconds
* @return random value
*/
public static int getRandomValueFromInterval(double randomizationFactor, double random, int currentIntervalMillis) {
double delta = randomizationFactor * currentIntervalMillis;
double minInterval = currentIntervalMillis - delta;
double maxInterval = currentIntervalMillis + delta;
// Get a random value from the range [minInterval, maxInterval].
// The formula used below has a +1 because if the minInterval is 1 and the maxInterval is 3 then
// we want a 33% chance for selecting either 1, 2 or 3.
return (int) (minInterval + (random * (maxInterval - minInterval + 1)));
}
/**
* Returns the initial retry interval in milliseconds.
* @return interval milliseconds
*/
public final int getInitialIntervalMillis() {
return initialIntervalMillis;
}
/**
* Returns the randomization factor to use for creating a range around the retry interval.
* @return randomization factor
* <p>
* A randomization factor of 0.5 results in a random period ranging between 50% below and 50%
* above the retry interval.
* </p>
*/
public final double getRandomizationFactor() {
return randomizationFactor;
}
/**
* Returns the current retry interval in milliseconds.
* @return current interval in milliseconds
*/
public final int getCurrentIntervalMillis() {
return currentIntervalMillis;
}
/**
* Returns the value to multiply the current interval with for each retry attempt.
* @return multiplier
*/
public final double getMultiplier() {
return multiplier;
}
/**
* Returns the maximum value of the back off period in milliseconds. Once the current interval
* reaches this value it stops increasing.
* @return maximum interval value in milliseconds
*/
public final int getMaxIntervalMillis() {
return maxIntervalMillis;
}
/**
* Returns the maximum elapsed time in milliseconds.
* @return maximum elapsed time in milliseconds
* <p>
* If the time elapsed since an {@link ExponentialBackOff} instance is created goes past the
* max_elapsed_time then the method {@link #nextBackOffMillis()} starts returning
* {@link BackOff#STOP}. The elapsed time can be reset by calling {@link #reset()}.
* </p>
*/
public final int getMaxElapsedTimeMillis() {
return maxElapsedTimeMillis;
}
/**
* Returns the elapsed time in milliseconds since an {@link ExponentialBackOff} instance is
* created and is reset when {@link #reset()} is called.
* @return the elapsed time in milliseconds
* <p>
* The elapsed time is computed using {@link System#nanoTime()}.
* </p>
*/
public final long getElapsedTimeMillis() {
return (nanoClock.nanoTime() - startTimeNanos) / 1000000;
}
/**
* Increments the current interval by multiplying it with the multiplier.
*/
private void incrementCurrentInterval() {
// Check for overflow, if overflow is detected set the current interval to the max interval.
if (currentIntervalMillis >= maxIntervalMillis / multiplier) {
currentIntervalMillis = maxIntervalMillis;
} else {
currentIntervalMillis *= multiplier;
}
}
/**
* Nano clock which can be used to measure elapsed time in nanoseconds.
*
* <p>
* The default system implementation can be accessed at {@link #SYSTEM}. Alternative implementations
* may be used for testing.
* </p>
*
*/
public interface NanoClock {
/**
* Returns the current value of the most precise available system timer, in nanoseconds for use to
* measure elapsed time, to match the behavior of {@link System#nanoTime()}.
* @return value of timer in nanoseconds
*/
long nanoTime();
/**
* Provides the default System implementation of a nano clock by using {@link System#nanoTime()}.
*/
NanoClock SYSTEM = System::nanoTime;
}
/**
* Builder for {@link ExponentialBackOff}.
*
* <p>
* Implementation is not thread-safe.
* </p>
*/
public static class Builder {
/** The initial retry interval in milliseconds. */
private int initialIntervalMillis = DEFAULT_INITIAL_INTERVAL_MILLIS;
/**
* The randomization factor to use for creating a range around the retry interval.
*
* <p>
* A randomization factor of 0.5 results in a random period ranging between 50% below and 50%
* above the retry interval.
* </p>
*/
private double randomizationFactor = DEFAULT_RANDOMIZATION_FACTOR;
/**
* The value to multiply the current interval with for each retry attempt.
*/
private double multiplier = DEFAULT_MULTIPLIER;
/**
* The maximum value of the back off period in milliseconds. Once the retry interval reaches
* this value it stops increasing.
*/
private int maxIntervalMillis = DEFAULT_MAX_INTERVAL_MILLIS;
/**
* The maximum elapsed time in milliseconds after instantiating {@link ExponentialBackOff} or
* calling {@link #reset()} after which {@link #nextBackOffMillis()} returns
* {@link BackOff#STOP}.
*/
private int maxElapsedTimeMillis = DEFAULT_MAX_ELAPSED_TIME_MILLIS;
/**
* Nano clock.
*/
private NanoClock nanoClock = NanoClock.SYSTEM;
public Builder() {
}
/**
* Builds a new instance of {@link ExponentialBackOff}.
* @return an {@link ExponentialBackOff} instance
*/
public ExponentialBackOff build() {
if (initialIntervalMillis <= 0) {
throw new IllegalArgumentException();
}
if (!(0 <= randomizationFactor && randomizationFactor < 1)) {
throw new IllegalArgumentException();
}
if (multiplier < 1) {
throw new IllegalArgumentException();
}
if ((maxIntervalMillis < initialIntervalMillis)) {
throw new IllegalArgumentException();
}
if (maxElapsedTimeMillis <= 0) {
throw new IllegalArgumentException();
}
return new ExponentialBackOff(this);
}
/**
* Sets the initial retry interval in milliseconds. The default value is
* {@link #DEFAULT_INITIAL_INTERVAL_MILLIS}. Must be {@code > 0}.
* @param initialIntervalMillis interval milliseconds
* @return the builder
*
* <p>
* Overriding is only supported for the purpose of calling the super implementation and changing
* the return type, but nothing else.
* </p>
*/
public Builder setInitialIntervalMillis(int initialIntervalMillis) {
this.initialIntervalMillis = initialIntervalMillis;
return this;
}
/**
* Sets the randomization factor to use for creating a range around the retry interval. The
* default value is {@link #DEFAULT_RANDOMIZATION_FACTOR}. Must fall in the range
* {@code 0 <= randomizationFactor < 1}.
* @param randomizationFactor the randomization factor
* @return the builder
*
* <p>
* A randomization factor of 0.5 results in a random period ranging between 50% below and 50%
* above the retry interval.
* </p>
*
* <p>
* Overriding is only supported for the purpose of calling the super implementation and changing
* the return type, but nothing else.
* </p>
*/
public Builder setRandomizationFactor(double randomizationFactor) {
this.randomizationFactor = randomizationFactor;
return this;
}
/**
* Sets the value to multiply the current interval with for each retry attempt. The default
* value is {@link #DEFAULT_MULTIPLIER}. Must be {@code >= 1}.
* @param multiplier the multiplier
* @return the builder
*
* <p>
* Overriding is only supported for the purpose of calling the super implementation and changing
* the return type, but nothing else.
* </p>
*/
public Builder setMultiplier(double multiplier) {
this.multiplier = multiplier;
return this;
}
/**
* Sets the maximum value of the back off period in milliseconds. Once the current interval
* reaches this value it stops increasing. The default value is
* {@link #DEFAULT_MAX_INTERVAL_MILLIS}.
* @param maxIntervalMillis maximum interval in miliseconds
* @return the builder
*
* <p>
* Overriding is only supported for the purpose of calling the super implementation and changing
* the return type, but nothing else.
* </p>
*/
public Builder setMaxIntervalMillis(int maxIntervalMillis) {
this.maxIntervalMillis = maxIntervalMillis;
return this;
}
/**
* Sets the maximum elapsed time in milliseconds. The default value is
* {@link #DEFAULT_MAX_ELAPSED_TIME_MILLIS}. Must be {@code > 0}.
* @param maxElapsedTimeMillis maximum elapsed time millis
* @return the builder
*
* <p>
* If the time elapsed since an {@link ExponentialBackOff} instance is created goes past the
* max_elapsed_time then the method {@link #nextBackOffMillis()} starts returning
* {@link BackOff#STOP}. The elapsed time can be reset by calling {@link #reset()}.
* </p>
*
* <p>
* Overriding is only supported for the purpose of calling the super implementation and changing
* the return type, but nothing else.
* </p>
*/
public Builder setMaxElapsedTimeMillis(int maxElapsedTimeMillis) {
this.maxElapsedTimeMillis = maxElapsedTimeMillis;
return this;
}
/**
* Sets the nano clock ({@link NanoClock#SYSTEM} by default).
* @param nanoClock the nano clock
* @return the builder
* <p>
* Overriding is only supported for the purpose of calling the super implementation and changing
* the return type, but nothing else.
* </p>
*/
public Builder setNanoClock(NanoClock nanoClock) {
if (nanoClock != null) {
this.nanoClock = nanoClock;
}
return this;
}
}
}

Some files were not shown because too many files have changed in this diff Show more