Custom Connector Code

  • Open the project

    In the Cloud9 IDE on the left hand side expand the aws-athena-query-federation project and navigate to the athena-example project and open file ExampleMetadataHandler.java file. Double Click on the file and it should open up for editing as shown here:
  • Update Code

  • In this file we have hard coded schemas to help learn how to write a connector and then you can apply that and interface with the target systems you ultimately want to federate to.

    Complete the TODOs in ExampleMetadataHandler by uncommenting the provided example code and providing missing code where indicated.
    Complete the TODOs in ExampleRecordHandler by uncommenting the provided example code and providing missing code where indicated.
        @Override
        public ListSchemasResponse doListSchemaNames(BlockAllocator allocator, ListSchemasRequest request)
        {
            logger.info("doListSchemaNames: enter - " + request);
    
            Set<String> schemas = new HashSet<>();
    
            /**
             * TODO: Add schemas, example below
             *
             schemas.add("schema1");
             *
             */
    
            return new ListSchemasResponse(request.getCatalogName(), schemas);
        }
    
            
    The modified UDF code can be copied by issuing the following commands:
            
            
            curl https://aws-data-analytics-workshops.s3.amazonaws.com/athena-workshop/scripts/ExampleMetadataHandler.java > athena-example/src/main/java/com/amazonaws/connectors/athena/example/ExampleMetadataHandler.java
    
        
    Once the file is copied, you can open it in Cloud9 IDE to see its contents. Do the same for the next files: ExampleRecordHandler.java and ExampleUserDefinedFuncHandler.java
            
            
            curl https://aws-data-analytics-workshops.s3.amazonaws.com/athena-workshop/scripts/ExampleRecordHandler.java > athena-example/src/main/java/com/amazonaws/connectors/athena/example/ExampleRecordHandler.java
    
             curl https://aws-data-analytics-workshops.s3.amazonaws.com/athena-workshop/scripts/ExampleUserDefinedFuncHandler.java > athena-example/src/main/java/com/amazonaws/connectors/athena/example/ExampleUserDefinedFuncHandler.java
    
        
  • Build the JAR File
  • Save the file and run mvn clean install to build your project. After it successfully builds, a JAR file is created in the target folder of your project named artifactId-version.jar, where artifactId is the name you provided in the Maven project, for example, athena-udfs.
            
            cd ~/environment/aws-athena-query-federation/athena-example/
    
            mvn clean install
        
  • Deploying The Connector

    From the athena-example dir, run ../tools/publish.sh S3_BUCKET_NAME athena-example to publish the connector to your private AWS Serverless Application Repository. The S3_BUCKET in the command is where a copy of the connector's code will be stored for Serverless Application Repository to retrieve it. This will allow users with permission to do so, the ability to deploy instances of the connector via 1-Click form.
            
            ../tools/publish.sh S3_BUCKET_NAME athena-example
        
    S3_BUCKET_NAME can be obtained from CloudFormation Output Key S3Bucket as shown here: When you run the command, it will look like this: Once the connector is published successfully, it will look like this: Click on the link shown in terminal after the publish is successful or alternatively navigate to Serverless Application Repository Click on Available Applications -> Private applications.