Amazon DynamoDB Local - 未知错误，异常或失败

3 浏览
0 Comments

Amazon DynamoDB Local - 未知错误，异常或失败

我需要测试一个严重依赖于亚马逊的DynamoDB的应用程序。我希望能够分别运行这些测试，这就是为什么我选择了DynamoDB Local .jar。我知道最近的更新使我们能够在不调用外部bash命令的情况下运行它。然而，当我尝试运行这里指定的示例时，我得到以下堆栈跟踪：

Exception in thread "main" com.amazonaws.AmazonServiceException: 由于未知错误、异常或故障，请求处理失败。（服务：AmazonDynamoDBv2；状态码：500；错误代码：InternalFailure；请求ID：cab7a550-aaa6-4bfe-a591-0b255481cc14）
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1275)
    at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:873)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:576)
    at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:362)
    at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:328)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:307)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:1805)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.listTables(AmazonDynamoDBClient.java:1223)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.listTables(AmazonDynamoDBClient.java:1235)

这是我尝试运行的代码：

public static void main( String[] args ) throws Exception
    {
        AmazonDynamoDB dynamodb = null;
        DynamoDBProxyServer server = null;
        final String[] localArgs = { "-inMemory", "-port", "13005" };
        server = ServerRunner.createServerFromCommandLineArgs(localArgs);
        server.start();
        BasicAWSCredentials auth = new BasicAWSCredentials("key", "secret");
        dynamodb = new AmazonDynamoDBClient(auth);
        dynamodb.setEndpoint("http://127.0.0.1:13005");
        // use the DynamoDB API over HTTP
        System.out.println(dynamodb.listTables());
        // Stop the DynamoDB Local endpoint
        if(server != null) {
            server.stop();
        }
    }

我观察到，如果我尝试完全从Java程序本身运行它，就会抛出异常，并且指定的端口不再可用（抛出错误，指出此端口被占用）。但是，如果我从命令提示符中启动DynamoDB Local，并将Java程序仅用作访问客户端，则一切都正常运行。

有什么建议吗？

0

2 答案

0 Comments

2018年8月，亚马逊宣布推出了带有Amazon DynamoDB Local的新的Docker镜像。使用这个镜像不再需要下载和运行任何JAR文件，也不需要使用第三方特定操作系统的二进制文件（我指的是sqlite4java）。

使用Docker容器非常简单，只需要在运行测试之前启动一个Docker容器即可：

docker run -p 8000:8000 amazon/dynamodb-local

你可以在本地开发时手动执行上述操作。IDE通常提供在执行任务之前运行任意命令的方法，所以你可以让IDE为你启动容器。

或者你可以在CI流水线中使用它。许多CI服务提供在流水线期间启动额外容器的能力，这些容器可以为你的测试提供依赖项。以下是Gitlab CI/CD的一个示例：

test:

stage: test

image: openjdk:8-alpine

services:

- name: amazon/dynamodb-local

alias: dynamodb-local

script:

- DYNAMODB_LOCAL_URL=http://dynamodb-local:8000 ./gradlew clean test

或者Bitbucket Pipelines的示例：

definitions:

services:

dynamodb-local:

image: amazon/dynamodb-local

…

step:

name: test

image:

name: openjdk:8-alpine

services:

- dynamodb-local

script:

- DYNAMODB_LOCAL_URL=http://localhost:8000 ./gradlew clean test

0
0 Comments

Amazon DynamoDB Local - 未知错误、异常或故障

在使用DynamoDBLocal时，至少存在两个问题。解决这两个问题，您将能够运行内嵌的DynamoDB。

首先，"-port"参数不起作用。因此，Jetty没有在您期望的端口上进行设置。而是将类似51205（或随机值？）设置为默认端口的Jetty监听器。

下面是我启动服务器的代码，避免使用内置的命令行解析，这种方式更好...以这种方式启动服务器后，"http://localhost:19444/shell"可以正常工作，所以Jetty没问题。但是，您可能会遇到另一个关于Sqlite4java的问题（请参见代码块后面的说明）。

注意：代码是Kotlin编写的，但Java的写法非常类似。此外，我已配置了SLF4j，并且不会让ServerRunner类破坏记录日志，所以这是启动服务器的更好方式，也是ServerRunner在内部执行的方式。

class TestAccountManager {

companion object {

private val localDbPort = 19444

private lateinit var localDb: DynamoDBProxyServer

private lateinit var dbClient: AmazonDynamoDBClient

fun setup() {

System.setProperty("org.eclipse.jetty.util.log.class", "org.eclipse.jetty.util.log.Slf4jLog")

localDb = DynamoDBProxyServer(localDbPort, LocalDynamoDBServerHandler(

LocalDynamoDBRequestHandler(0, true, null, true, true), null)

)

localDb.start()

val auth = BasicAWSCredentials("fakeKey", "fakeSecret")

dbClient = AmazonDynamoDBClient(auth)

dbClient.signerRegionOverride = "us-east-1"

dbClient.setEndpoint("http://localhost:$localDbPort")

}

fun teardown() {

localDb.stop()

}

}

fun testSomething() {

dbClient.listTables().tableNames.forEach {

println(it)

}

}

}

使用这段代码，您现在可以在预期的端口上运行了。

现在，您可能会遇到第二个错误，例如Sqlite4java无法找到适用于您平台的正确二进制文件。对于某些Mac OSX版本，它将生成一个实际上不存在的二进制文件名。而且DynamoDBLocal强制隐藏了所有Sqlite4java的日志记录（无法覆盖它），因此您无法看到它。

您可以通过下载一个分发版来测试sqlite库，然后解压并运行：

java -jar sqlite4java-1.0.392.jar -d

它将报告它尝试加载的内容以及加载是否失败。您只需要从Gradle、Maven或其他位置在系统中找到该JAR文件。我遇到的错误输出如下：

sqlite4java 392

160212:002049.833 FINE [sqlite] Internal: loading library

160212:002049.834 FINE [sqlite] Internal: java.library.path=/Users/jminard/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.

160212:002049.834 FINE [sqlite] Internal: sqlite4java.library.path=null

160212:002049.834 FINE [sqlite] Internal: cwd=/Users/jminard/DEV/Collokia/repos/collokia-web-back/.

160212:002049.834 FINE [sqlite] Internal: default path=/Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d

160212:002049.834 FINE [sqlite] Internal: forced path=null

160212:002049.834 FINE [sqlite] Internal: os.name=mac os x; os=osx

160212:002049.835 FINE [sqlite] Internal: os.arch=x86_64

160212:002049.835 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-x86_64-1.0.392.dylib

160212:002049.835 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-amd64-1.0.392.dylib

160212:002049.835 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-1.0.392.dylib

160212:002049.835 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-1.0.392.dylib

160212:002049.836 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-x86_64-d-1.0.392.dylib

160212:002049.836 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-amd64-d-1.0.392.dylib

160212:002049.836 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-d-1.0.392.dylib

160212:002049.836 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-d-1.0.392.dylib

160212:002049.836 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-x86_64.dylib

160212:002049.836 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-amd64.dylib

160212:002049.836 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx.dylib

160212:002049.837 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java.dylib

160212:002049.837 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-x86_64-d.dylib

160212:002049.837 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-amd64-d.dylib

160212:002049.837 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-osx-d.dylib

160212:002049.837 FINE [sqlite] Internal: checking /Users/jminard/.gradle/caches/modules-2/files-2.1/com.almworks.sqlite4java/sqlite4java/1.0.392/d6234e08ff4e1607ff5321da2579571f05ff778d/libsqlite4java-d.dylib

160212:002049.837 FINE [sqlite] Internal: trying to load sqlite4java-osx-x86_64-1.0.392

160212:002049.838 FINE [sqlite] Internal: cannot load sqlite4java-osx-x86_64-1.0.392: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-x86_64-1.0.392 in java.library.path

160212:002049.838 FINE [sqlite] Internal: trying to load sqlite4java-osx-amd64-1.0.392

160212:002049.839 FINE [sqlite] Internal: cannot load sqlite4java-osx-amd64-1.0.392: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-amd64-1.0.392 in java.library.path

160212:002049.839 FINE [sqlite] Internal: trying to load sqlite4java-osx-1.0.392

160212:002049.840 FINE [sqlite] Internal: cannot load sqlite4java-osx-1.0.392: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-1.0.392 in java.library.path

160212:002049.840 FINE [sqlite] Internal: trying to load sqlite4java-1.0.392

160212:002049.841 FINE [sqlite] Internal: cannot load sqlite4java-1.0.392: java.lang.UnsatisfiedLinkError: no sqlite4java-1.0.392 in java.library.path

160212:002049.841 FINE [sqlite] Internal: trying to load sqlite4java-osx-x86_64-d-1.0.392

160212:002049.842 FINE [sqlite] Internal: cannot load sqlite4java-osx-x86_64-d-1.0.392: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-x86_64-d-1.0.392 in java.library.path

160212:002049.842 FINE [sqlite] Internal: trying to load sqlite4java-osx-amd64-d-1.0.392

160212:002049.842 FINE [sqlite] Internal: cannot load sqlite4java-osx-amd64-d-1.0.392: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-amd64-d-1.0.392 in java.library.path

160212:002049.843 FINE [sqlite] Internal: trying to load sqlite4java-osx-d-1.0.392

160212:002049.843 FINE [sqlite] Internal: cannot load sqlite4java-osx-d-1.0.392: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-d-1.0.392 in java.library.path

160212:002049.843 FINE [sqlite] Internal: trying to load sqlite4java-d-1.0.392

160212:002049.844 FINE [sqlite] Internal: cannot load sqlite4java-d-1.0.392: java.lang.UnsatisfiedLinkError: no sqlite4java-d-1.0.392 in java.library.path

160212:002049.844 FINE [sqlite] Internal: trying to load sqlite4java-osx-x86_64

160212:002049.845 FINE [sqlite] Internal: cannot load sqlite4java-osx-x86_64: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-x86_64 in java.library.path

160212:002049.845 FINE [sqlite] Internal: trying to load sqlite4java-osx-amd64

160212:002049.845 FINE [sqlite] Internal: cannot load sqlite4java-osx-amd64: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-amd64 in java.library.path

160212:002049.845 FINE [sqlite] Internal: trying to load sqlite4java-osx

160212:002049.846 FINE [sqlite] Internal: cannot load sqlite4java-osx: java.lang.UnsatisfiedLinkError: no sqlite4java-osx in java.library.path

160212:002049.846 FINE [sqlite] Internal: trying to load sqlite4java

160212:002049.847 FINE [sqlite] Internal: cannot load sqlite4java: java.lang.UnsatisfiedLinkError: no sqlite4java in java.library.path

160212:002049.847 FINE [sqlite] Internal: trying to load sqlite4java-osx-x86_64-d

160212:002049.848 FINE [sqlite] Internal: cannot load sqlite4java-osx-x86_64-d: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-x86_64-d in java.library.path

160212:002049.848 FINE [sqlite] Internal: trying to load sqlite4java-osx-amd64-d

160212:002049.849 FINE [sqlite] Internal: cannot load sqlite4java-osx-amd64-d: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-amd64-d in java.library.path

160212:002049.849 FINE [sqlite] Internal: trying to load sqlite4java-osx-d

160212:002049.849 FINE [sqlite] Internal: cannot load sqlite4java-osx-d: java.lang.UnsatisfiedLinkError: no sqlite4java-osx-d in java.library.path

160212:002049.850 FINE [sqlite] Internal: trying to load sqlite4java-d

160212:002049.850 FINE [sqlite] Internal: cannot load sqlite4java-d: java.lang.UnsatisfiedLinkError: no sqlite4java-d in java.library.path

Error: cannot load SQLite

java.lang.UnsatisfiedLinkError: no sqlite4java-osx-x86_64-1.0.392 in java.library.path

at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1864)

at java.lang.Runtime.loadLibrary0(Runtime.java:870)

at java.lang.System.loadLibrary(System.java:1122)

at com.almworks.sqlite4java.Internal.tryLoadFromSystemPath(Internal.java:352)

at com.almworks.sqlite4java.Internal.loadLibraryX(Internal.java:124)

at com.almworks.sqlite4java.SQLite.main(SQLite.java:368)

如果您遇到关于动态库加载的错误，请阅读以下链接以解决问题：

- [UnsatisfiedLinkError with sqlite4java Jar on Mac OS X NetBeans](https://stackoverflow.com/questions/15559551)

- [UnsatisfiedLinkError with sqlite4java Jar on Mac OS X](https://stackoverflow.com/questions/14301562)

- 特别是与DynamoDBLocal相关的问题：[Run DynamoDB Local with the java command on Mac OS X](https://stackoverflow.com/questions/24894109)

- [https://groups.google.com/forum/#!topic/sqlite4java/9J1lmCuoKLA](https://groups.google.com/forum/#!topic/sqlite4java/9J1lmCuoKLA)

- [https://groups.google.com/forum/#!topic/sqlite4java/jhwt44nYGvw](https://groups.google.com/forum/#!topic/sqlite4java/jhwt44nYGvw)

似乎唯一可靠的解决方法是：

- 将从下载的[sqlite4java分发版](https://bitbucket.org/almworks/sqlite4java)中的库添加到已知位置，并设置系统属性`java.library.path`。

- 或者将从下载的[sqlite4java分发版](https://bitbucket.org/almworks/sqlite4java)中的库添加到`/Library/Java/Extensions`（仅适用于macOS）。

我通常采用第一种选项，将库添加到项目中，并确保构建时添加`-Djava.library.path=./lib/sqlite4java`，其中动态库被解压缩的位置。为了更可靠的测试，您可以在代码中以编程方式设置`java.library.path`，使用这个技巧（否则，在代码中设置时将被忽略）：[http://blog.cedarsoft.com/2010/11/setting-java-library-path-programmatically/](http://blog.cedarsoft.com/2010/11/setting-java-library-path-programmatically/)

DynamoDBLocal隐藏了所有的Sqlite错误，因此您必须进行调试以找到静默失败。该库中充满了调整日志级别的内容，使得调试变得困难，因为它们破坏了您查看错误的能力。例如，每当打开SqlLite文件时（类`SQLiteDBAccess`）：

LocalDBUtils.setLog4jToUtilsLogging("com.almworks.sqlite4java");

LocalDBUtils.setLog4jToUtilsLogging("com.almworks.sqlite4java.Internal");

java.util.logging.Logger.getLogger("com.almworks.sqlite4java").setLevel(Level.OFF);

java.util.logging.Logger.getLogger("com.almworks.sqlite4java.Internal").setLevel(Level.OFF);

尾注：我正在寻找替代方案，这不是地球上构建得最好的东西，查看DynamoDbLocal的代码和为其做出的决策对我来说没有给予太多信心。Alternator或Jcabi - Dynamo Mock是我接下来要尝试的。

0