Amazon s3 sdk for java2.x 最新api的简单使用

分类:热门推荐日期:2024-03-16浏览:1评论:0

目录

创建存储桶

登录亚马逊控制台 查找服务“S3” 进入存储桶创建页面；

点击 “创建存储桶”进入创建页面；

获取密钥ID和密钥

点击右上方账户信息在下拉菜单中选择 “安全凭证”（Security credentials）进入我的安全凭证页面；

下滑找到 “访问密钥”标签  点击右侧  “创建访问密钥”按钮；

步骤一： “访问密钥最佳实践和替代方案” 选择适合自己的方案

步骤二：“设置描述标签”可不填

步骤三 “检索访问密钥“

关闭屏蔽公共访问权限

点击创建的存储桶名称进入详情页面后点击“权限”标签

找到 “屏蔽公共访问权限” 点击下方 “编辑” 按钮

取消勾选“阻止所有公开访问”然后点击下方 “保存更改” 按钮；在弹出的确认框中输入“确认”后点击“确认”按钮即可生效。

创建存储桶访问策略

点击创建的存储桶名称进入详情页面后点击“权限”标签

找到存储桶策略 -> 点击右侧“编辑”

创建如下访问策略

代码示例

        首先导入依赖

        工具类

国内使用s3的坑点: 

亚马逊中国区，必须进行ICP 备案，申请开发80或443端口，否则是不能通过网址直接访问s3内部对象的。

备注：以下操作基于您已有AWS账户；

需要获取：存储桶名称、所选区域、密钥ID和密钥；

创建存储桶

  • 登录亚马逊控制台 查找服务“S3” 进入存储桶创建页面；

    点击 “创建存储桶”进入创建页面；

    1. 输入存储桶名称 、选择区域；

            2. 其他设置保持默认即可；

            3. 点击 “创建存储桶”,即可完成存储桶的创建；

    获取密钥ID和密钥

    点击右上方账户信息在下拉菜单中选择 “安全凭证”（Security credentials）进入我的安全凭证页面；

    下滑找到 “访问密钥”标签  点击右侧  “创建访问密钥”按钮；

    1. 进入创建访问密钥之后

    步骤一： “访问密钥最佳实践和替代方案” 选择适合自己的方案

    •  勾选下方复选框
    • 点击下一步；

      步骤二：“设置描述标签”可不填

      • 点击 “创建访问密钥”按钮即可创建密钥；

        步骤三 “检索访问密钥“

        1. 点击“下载.csv文件“ 保存密钥ID和密钥
        2. 注意：请保存好下载的.csv文件，离开此页面后“密钥”将没有地方获取查看；

        关闭屏蔽公共访问权限

        点击创建的存储桶名称进入详情页面后点击“权限”标签

        找到 “屏蔽公共访问权限” 点击下方 “编辑” 按钮

        取消勾选“阻止所有公开访问”然后点击下方 “保存更改” 按钮；在弹出的确认框中输入“确认”后点击“确认”按钮即可生效。

        创建存储桶访问策略

        点击创建的存储桶名称进入详情页面后点击“权限”标签

        1. 找到存储桶策略 -> 点击右侧“编辑”

        1. 创建如下访问策略

        {
    "Version": "2012-10-17",
    "Id": "GetImgAndPdf",
    "Statement": [
        {
            "Sid": "GetImgAndPdf123",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws-cn:s3:::所创建存储桶名称/*"
        }
    ]
}

        代码示例

                首先导入依赖

             software.amazon.awssdk bom ${awsjavasdk.version} pom import     software.amazon.awssdk s3   software.amazon.awssdk kms   software.amazon.awssdk s3control   software.amazon.awssdk s3-transfer-manager   software.amazon.awssdk.crt aws-crt 0.24.0

                工具类

        package com.trainingos.file.utils;
import com.trainingos.common.util.StringUtils;
import com.trainingos.common.util.file.FileTypeUtils;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.io.FilenameUtils;
import org.apache.http.entity.ContentType;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.web.multipart.MultipartFile;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.services.s3.*;
import software.amazon.awssdk.services.s3.model.*;
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;
import java.io.*;
import java.nio.charset.Charset;
import java.nio.file.Path;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import java.util.zip.ZipInputStream;
/**
 * @Description 亚马逊文件上传工具类
 * @Author ajie
 **/
@Slf4j
public class FileS3Utils {
/**
     * 密钥ID
     */
    @Value("${aws.s3.accessKeyId}")
    private String accessKeyId;
    /**
     * 密钥
     */
    @Value("${aws.s3.secretKey}")
    private String secretKey;
    /*@Value("${aws.s3.s3Uri}")
    private String s3Uri;*/
    /**
     * 存储桶名字
     */
    @Value("${aws.s3.bucket}")
    private String bucketName;
    /**
     * 地区节点
     */
    @Value("${aws.s3.region}")
    private String regionCode;
    /**
     * 资源映射前缀
     */
    @Value("${aws.s3.path}")
    private String filePathPrefix;
    public static String S3_ACCESS_KEY_ID = null;
    public static String S3_SECRET_KEY = null;
    //public static String S3_URI = null;
    public static String S3_BUCKET = null;
    public static String S3_REGION = null;
    public static String S3_PATH_PREFIX = null;
    /**
     * 初始化
     */
    @PostConstruct
    public void init() {
        S3_ACCESS_KEY_ID = accessKeyId;
        S3_SECRET_KEY = secretKey;
        //S3_URI = s3Uri;
        S3_BUCKET = bucketName;
        S3_REGION = regionCode;
        S3_PATH_PREFIX = filePathPrefix;
    }
    /**
     * S3客户端对象
     */
    private static S3Client s3Client;
    public static S3TransferManager transferManager;
    /**
     * 预签名对象
     */
    private static S3Presigner s3Presigner;
    /**
     * 获取S3客户端对象
     *
     * @return S3Client
     */
    public static synchronized S3Client getS3Client() {
        if (null == s3Client) {
            s3Client = S3Client.builder()
                    .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(S3_ACCESS_KEY_ID, S3_SECRET_KEY)))
                    //.endpointConfiguration(new AwsClientBuilder.EndpointConfiguration(hostName, region))  // 如果有endpoint，可以用这个，这个和withRegion(Region)不能一起使用
                    //.withPathStyleAccessEnabled(true)  // 如果配置了S3域名，就需要加这个进行路径访问，要不然会报AccessKey不存在的问题
                    .region(Region.of(S3_REGION))
                    .build();
        }
        return s3Client;
    }
    /**
     * 上传大文件 高级API
     * @return
     */
    private static synchronized S3TransferManager getTransferManager(){
        if(null == transferManager){
            S3AsyncClient s3AsyncClient =
                    S3AsyncClient.crtBuilder()
                            .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(S3_ACCESS_KEY_ID, S3_SECRET_KEY)))
                            .region(Region.of(S3_REGION))
                            //.targetThroughputInGbps(20.0)
                            //.minimumPartSizeInBytes(8 * MB)
                            .build();
            transferManager =
                    S3TransferManager.builder()
                            .s3Client(s3AsyncClient)
                            .build();
        }
        return transferManager;
    }
    /**
     * 获取预签名对象
     *
     * @return
     */
    public static synchronized S3Presigner getS3PreSigner() {
        if (null == s3Presigner) {
            s3Presigner = S3Presigner.builder()
                    // 凭证
                    .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(S3_ACCESS_KEY_ID, S3_SECRET_KEY)))
                    //.endpointOverride(URI.create(S3_URI))
                    // 服务配置
                    //.serviceConfiguration(S3Configuration.builder().checksumValidationEnabled(false).pathStyleAccessEnabled(true).build())
                    // 区域
                    .region(Region.of(S3_REGION))
                    .build();
        }
        return s3Presigner;
    }
    /**
     * 上传文件
     *
     * @param s3
     * @param bucketName
     * @param awsFilePrefix aws资源映射前缀
     * @param file
     * @param catalog
     * @return
     * @throws IOException
     */
    public static String uploadPublicFile(S3Client s3, String bucketName, String awsFilePrefix, MultipartFile file, String catalog) throws IOException {
        // 生成新文件名
        String fileName = FileUploadUtils.extractFilename(file);
        try {
            String keyName = awsFilePrefix + catalog + "/" + fileName;
            log.info("keyName===》 {} ", keyName);
            String contentType = FileTypeUtils.getFileType(fileName);
            log.info("文件类型===》 {} ", contentType);
            // 使用PutObjectRequest来设置附加的值
            PutObjectRequest putObjectRequest = PutObjectRequest.builder()
                    .bucket(bucketName)
                    .key(keyName)
                    .contentType(contentType)
                    .contentLength(file.getSize())
                    .acl(ObjectCannedACL.PUBLIC_READ)
                    .build();
            // 上传文件
            s3.putObject(putObjectRequest, RequestBody.fromBytes(file.getBytes()));
            log.info("======上传成功=======》 {} ", catalog + "/" + fileName);
            return catalog + "/" + fileName;
        } catch (S3Exception | IOException e) {
            log.info("上传文件：", e);
            throw new IOException(e.getMessage(), e);
        }
    }
   
    /**
     * 预览 有效时间为1天
     *
     * @param key 文件地址  桶中文件全路径
     * @return
     */
    private static String getPreSignatureUrl(S3Presigner s3Presigner, String bucketName, String key) throws Exception {
        String preSignatureUrl = "";
        try {
            GetObjectRequest getObjectRequest = GetObjectRequest.builder()
                    .bucket(bucketName)
                    .key(key)
                    .build();
            //设置预签名URL可访问时间 1天
            GetObjectPresignRequest getObjectPresignRequest = GetObjectPresignRequest.builder()
                    .signatureDuration(Duration.ofDays(1))
                    .getObjectRequest(getObjectRequest)
                    .build();
            PresignedGetObjectRequest presignedGetObjectRequest = s3Presigner.presignGetObject(getObjectPresignRequest);
            preSignatureUrl = String.valueOf(presignedGetObjectRequest.url());
        } catch (Exception e) {
            log.info("生成预签名URL失败，异常: ", e);
            throw new Exception(e.getMessage(), e);
        }
        return preSignatureUrl;
    }
    /**
     * 检查对象是否存在
     *
     * @param s3Client s3客户端
     * @param key      文件在桶中的全路径地址
     * @return
     */
    public static boolean checkIfObjectExists(S3Client s3Client, String bucketName, String key) {
        try {
            HeadObjectRequest headObjectRequest = HeadObjectRequest.builder()
                    .bucket(bucketName)
                    .key(key)
                    .build();
            s3Client.headObject(headObjectRequest);
            return true; // 对象存在
        } catch (NoSuchKeyException e) {
            return false; // 对象不存在
        } catch (S3Exception e) {
            log.info("Error checking if object exists: ", e);
            return false; // 其他错误
        }
    }
    /**
     * 流的方式上传文件到s3桶
     *
     * @param s3Client
     * @param bucketName
     * @param key
     * @param file
     */
    public static void uploadFileStream(S3Client s3Client, String bucketName, String key, File file) {
        try {
            FileInputStream fileInputStream = new FileInputStream(file);
            PutObjectRequest objectRequest = PutObjectRequest.builder()
                    .bucket(bucketName)
                    .key(key)
                    .build();
            // 使用 RequestBody.fromInputStream 方法将文件流上传到 S3
            s3Client.putObject(objectRequest, RequestBody.fromInputStream(fileInputStream, file.length()));
            System.out.println("File uploaded to S3 successfully.");
        } catch (Exception e) {
            System.err.println("Error uploading file to S3: " + e.getMessage());
        }
    }
    /**
     * 获取桶中某目录下所有文件
     *
     * @param s3Client
     * @param bucketName
     * @param prefix     指定前缀，例如文件夹路径
     * @return
     */
    public static List listFilesInBucket(S3Client s3Client, String bucketName, String prefix) {
        ListObjectsV2Request listObjectsRequest = ListObjectsV2Request.builder()
                .bucket(bucketName)
                .prefix(prefix)
                .build();
        ListObjectsV2Response response = s3Client.listObjectsV2(listObjectsRequest);
        return response.contents();
    }
    /**
     * 上传对象
     */
//    public static void main(String[] args){
//
//        try{
//            Long start = System.currentTimeMillis();
//            String baseUrl = "upload/images/common/";
//
//            //File file = new File("C://Users//阿杰//Pictures//微信图片_20220106110701.png");
//            File file = new File("C://Users//阿杰//Videos//《历史关头——红色纪念馆之旅》第二集：星星之火，可以燎原.mp4");
//
//            boolean b = checkIfObjectExists(getS3Client(), "《历史关头——红色纪念馆之旅》第二集：星星之火，可以燎原.mp4");
//            if(b){
//                log.info("============文件已存在=====");
//                return;
//            }
//            //multipartUpload(baseUrl + "/《历史关头——红色纪念馆之旅》第二集：星星之火，可以燎原.mp4",file);
//
//            //MultipartFile multipartFile = getMultipartFile(file);
//
//            //String address = uploadPublicFile(baseUrl, multipartFile);
//            log.info("===========上传完成=====用时：{}",(double)(System.currentTimeMillis()-start)/1000);
//            //log.info("=========文件地址=====》 {} ", address);
//
//            //String upload = FileUploadUtils.upload("D:/wenjian/project" + File.separator + "png", multipartFile);
//            //log.info("=========文件地址=====》 {} ", upload);
//
//            log.info("================================================");
//
//            //String preview = getPreSignatureUrl(baseUrl + "2023/11/06/《历史关头——红色纪念馆之旅》第二集：星星之火，可以燎原_20231106150748A001.mp4");
//            //log.info("=========文件预览地址=====》 {} ", preview);
//
//            //getURL(baseUrl + "《历史关头——红色纪念馆之旅》第二集：星星之火，可以燎原.mp4");
//
//            //===================================================
//        }catch (Exception e){
//            log.info("============",e);
//        }
//    }
    /**
     * file 转 MultipartFile
     *
     * @param file
     * @return
     */
    public static MultipartFile converFileToMultipartFile(File file) throws IOException {
        InputStream inputStream = new FileInputStream(file);
        return new MockMultipartFile(file.getName(), file.getName(), ContentType.APPLICATION_OCTET_STREAM.toString(), inputStream);
    }

    /**
     * 转换 MultipartFile 为 File 对象
     *
     * @param multipartFile
     * @return
     * @throws IOException
     */
    public static File convertMultipartFileToFile(MultipartFile multipartFile) throws IOException {
        File file = new File(Objects.requireNonNull(multipartFile.getOriginalFilename()));
        try (OutputStream os = new FileOutputStream(file)) {
            os.write(multipartFile.getBytes());
        }
        return file;
    }
    private static final long PART_SIZE = 100 * 1024 * 1024; // 设置文件分段大小 100 M
    /**
     * 计算分段数量
     */
    public static long getPartNumber(long fileLength) {
        if (fileLength % PART_SIZE == 0) {
            return fileLength / PART_SIZE;
        } else {
            return fileLength / PART_SIZE + 1;
        }
    }
    private static void multipartUpload(S3Client s3Client, String bucketName, String objectKey, File file) {
        CreateMultipartUploadRequest createMultipartUploadRequest = CreateMultipartUploadRequest.builder()
                .bucket(bucketName)
                .key(objectKey)
                .build();
        CreateMultipartUploadResponse response = s3Client.createMultipartUpload(createMultipartUploadRequest);
        String uploadId = response.uploadId();
        try {
            FileInputStream fis = new FileInputStream(file);
            List completedParts = new ArrayList<>();
            final long fileLength = file.length();
            final long partNumber = getPartNumber(fileLength);
            log.info("multipartUpload fileLength={}, partNumber={},uploadId={}", fileLength, partNumber, uploadId);
            for (int i = 1; i <= partNumber; i++) {
                final byte[] bytes = fis.readNBytes((int) PART_SIZE);
                //String md5 = new String(Base64.encodeBase64(DigestUtils.md5(bytes)));
                UploadPartRequest uploadPartRequest = UploadPartRequest.builder()
                        .bucket(bucketName)
                        .key(objectKey)
                        .uploadId(uploadId)
                        .partNumber(i)
                        //.contentMD5(md5)
                        .build();
                final RequestBody requestBody = RequestBody.fromBytes(bytes);
                UploadPartResponse uploadPartResponse = s3Client.uploadPart(uploadPartRequest, requestBody);
                String eTag = uploadPartResponse.eTag();
                CompletedPart part = CompletedPart.builder().partNumber(i).eTag(eTag).build();
                completedParts.add(part);
            }
            CompletedMultipartUpload completedMultipartUpload = CompletedMultipartUpload.builder()
                    .parts(completedParts)
                    .build();
            CompleteMultipartUploadRequest completeMultipartUploadRequest =
                    CompleteMultipartUploadRequest.builder()
                            .bucket(bucketName)
                            .key(objectKey)
                            .uploadId(uploadId)
                            .multipartUpload(completedMultipartUpload)
                            .build();
            s3Client.completeMultipartUpload(completeMultipartUploadRequest);
        } catch (Exception e) {
            log.error("S3 multipartUpload fail!", e);
            // 停止正在进行的分段上传，清理已上传分段
            s3Client.abortMultipartUpload(AbortMultipartUploadRequest.builder()
                    .bucket(bucketName)
                    .key(objectKey)
                    .uploadId(uploadId)
                    .build());
        }
    }
    /*public static void main(String[] args) {
        Long start = System.currentTimeMillis();
        // 设置所在的AWS Region
        Region region = Region.of(S3_REGION); // 根据您的需求更改Region
        // 创建 S3 客户端
        S3Client s3 = S3Client.builder().credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(S3_ACCESS_KEY_ID, S3_SECRET_KEY))).region(region).build();
        // 定义所使用的存储桶和文件键值
        String bucketName = "nxbvideo"; // 请替换为您要使用的存储桶名称
        String key = "《历史关头——红色纪念馆之旅》第二集：星星之火，可以燎原.mp4"; // 请替换为您要上传的文件的键值
        // 定义本地文件路径
        Path filePath = Paths.get("C://Users//阿杰//Videos//《历史关头——红色纪念馆之旅》第二集：星星之火，可以燎原.mp4"); // 请替换为您要上传的本地文件路径
        // 开始分段上传
        String uploadId = initiateMultipartUpload(s3, bucketName, key);
        // 选择每个分段的大小（100MB），可以根据需要调整分段大小
        long partSize = 100 * 1024 * 1024;
        // 分割文件并上传每个部分
        List completedParts = uploadParts(s3, bucketName, key, filePath, partSize,uploadId);
        System.out.println("completedParts 分了==========>"+completedParts.size());
        // 完成分段上传
        completeMultipartUpload(s3, bucketName, key, uploadId, completedParts);
        log.info("===========上传完成=====用时：{}",(double)(System.currentTimeMillis()-start)/1000);
        // 关闭 S3 客户端
        s3.close();
    }*/
    // 初始化分段上传
    public static String initiateMultipartUpload(S3Client s3, String bucketName, String key) {
        CreateMultipartUploadResponse response = s3.createMultipartUpload(
                CreateMultipartUploadRequest.builder()
                        .bucket(bucketName)
                        .key(key)
                        .build()
        );
        return response.uploadId();
    }
    // 分割文件并上传每个部分
    public static List uploadParts(S3Client s3, String bucketName, String key, Path filePath, long partSize, String uploadId) {
        List completedParts = new ArrayList<>();
        try {
            long filePosition = 0;
            long contentLength = filePath.toFile().length();
            for (int partNumber = 1; filePosition < contentLength; partNumber++) {
                long remainingBytes = contentLength - filePosition;
                long byteCount = Math.min(partSize, remainingBytes);
                UploadPartRequest uploadPartRequest = UploadPartRequest.builder()
                        .bucket(bucketName)
                        .key(key)
                        .uploadId(uploadId)
                        .partNumber(partNumber)
                        .contentLength(byteCount)
                        .build();
                UploadPartResponse response = s3.uploadPart(
                        uploadPartRequest,
                        RequestBody.fromFile(filePath)
                );
                completedParts.add(CompletedPart.builder()
                        .partNumber(partNumber)
                        .eTag(response.eTag())
                        .build());
                filePosition += byteCount;
            }
            return completedParts;
        } catch (Exception e) {
            System.err.println("Multipart upload failed: " + e.getMessage());
            s3.abortMultipartUpload(AbortMultipartUploadRequest.builder()
                    .bucket(bucketName)
                    .key(key)
                    .uploadId(uploadId)
                    .build());
            return null;
        }
    }
    // 中止分段上传
    public static void abortMultipartUpload(S3Client s3, String bucketName, String key, String uploadId) {
        s3.abortMultipartUpload(AbortMultipartUploadRequest.builder()
                .bucket(bucketName)
                .key(key)
                .uploadId(uploadId)
                .build());
        System.out.println("Multipart upload aborted.");
    }
    // 续传分段
    public static void resumeMultipartUpload(S3Client s3, String bucketName, String key, Path filePath, long partSize, String uploadId, List completedParts) {
        // 重新初始化分段上传
        uploadId = initiateMultipartUpload(s3, bucketName, key);
        // 检查哪些分段已经上传
        long partNumber = completedParts.size() + 1;
        long filePosition = completedParts.stream()
                .mapToLong(part -> part.partNumber())
                .sum() * partSize;
        try {
            long contentLength = filePath.toFile().length();
            for (; filePosition < contentLength; partNumber++) {
                long remainingBytes = contentLength - filePosition;
                long byteCount = Math.min(partSize, remainingBytes);
                UploadPartResponse response = s3.uploadPart(
                        UploadPartRequest.builder()
                                .bucket(bucketName)
                                .key(key)
                                .uploadId(uploadId)
                                .partNumber((int) partNumber)
                                .contentLength(byteCount)
                                .build(),
                        RequestBody.fromFile(filePath)
                );
                completedParts.add(CompletedPart.builder()
                        .partNumber((int) partNumber)
                        .eTag(response.eTag())
                        .build());
                filePosition += byteCount;
            }
            // 完成续传的分段上传
            completeMultipartUpload(s3, bucketName, key, uploadId, completedParts);
        } catch (Exception e) {
            System.err.println("Multipart upload failed: " + e.getMessage());
            abortMultipartUpload(s3, bucketName, key, uploadId);
        }
    }
    // 完成分段上传
    public static void completeMultipartUpload(S3Client s3, String bucketName, String key, String uploadId, List completedParts) {
        CompleteMultipartUploadResponse response = s3.completeMultipartUpload(
                CompleteMultipartUploadRequest.builder()
                        .bucket(bucketName)
                        .key(key)
                        .uploadId(uploadId)
                        .multipartUpload(CompletedMultipartUpload.builder().parts(completedParts).build())
                        .build()
        );
        System.out.println("Multipart upload completed. ETag: " + response.eTag());
        // 因为在分段上传过程中，单独的每个分段并不包含文件的完整 Metadata。
        // 在完成上传之后，使用 CopyObjectRequest 设置文件类型
        // 创建 Metadata 对象并设置文件类型
        /*Map metadata = new HashMap<>();
        metadata.put("Content-Type", "application/zip");
        CopyObjectRequest copyObjectRequest = CopyObjectRequest.builder()
                .copySource(bucketName + "/" + key)
                .destinationBucket(bucketName)
                .destinationKey(key)
                .metadataDirective(MetadataDirective.REPLACE) // 覆盖现有元数据
                .metadata(metadata)
                .build();
        s3.copyObject(copyObjectRequest);
        System.out.println("Multipart upload completed. Content-Type set to application/zip.");*/
    }
    // 分片上传
    /*public static void main(String[] args) {
        String bucketName = S3_BUCKET;
        String key = "upload/images/common/《历史关头——红色纪念馆之旅》第二集：星星之火，可以燎原.mp4"; // 文件在 S3 中的键
        String filePathString = "C://Users//阿杰//Videos//《历史关头——红色纪念馆之旅》第二集：星星之火，可以燎原.mp4"; // 本地文件路径
        // 创建 S3 客户端
        S3Client s3 = S3Client.builder()
                .region(Region.of(S3_REGION)) // 设置您的 S3 存储桶所在的区域
                .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(S3_ACCESS_KEY_ID, S3_SECRET_KEY)))
                .build();
        // 指定要上传的本地文件路径
        Path filePath = Path.of(filePathString);
        // 设置分段大小
        final long PART_SIZE = 50 * 1024 * 1024; // 每个分段的大小为 50MB
        // 创建 InitiateMultipartUploadRequest 请求
        CreateMultipartUploadRequest createRequest = CreateMultipartUploadRequest.builder()
                .bucket(bucketName)
                .key(key)
                .build();
        // 初始化分段上传
        CreateMultipartUploadResponse createResponse = s3.createMultipartUpload(createRequest);
        // 获取文件大小
        long contentLength = filePath.toFile().length();
        // 计算分段数量
        long partCount = (contentLength + PART_SIZE - 1) / PART_SIZE;
        // 创建一个线程池
        ExecutorService executorService = Executors.newFixedThreadPool(10);
        try {
            List> futures = new ArrayList<>();
            // 逐个上传文件的分段
            for (int i = 1; i <= partCount; i++) {
                long filePosition = (i - 1) * PART_SIZE;
                long partSize = Math.min(PART_SIZE, contentLength - filePosition);
                // 开启线程上传分段
                int finalI = i;
                Future future = executorService.submit(() -> {
                    UploadPartRequest uploadRequest = UploadPartRequest.builder()
                            .bucket(bucketName)
                            .key(key)
                            .uploadId(createResponse.uploadId())
                            .partNumber(finalI)
                            .build();
                    // 读取文件分段内容
                    RequestBody requestBody = RequestBody.fromFile(filePath);
                    // 上传分段
                    UploadPartResponse uploadPartResponse = s3.uploadPart(uploadRequest, requestBody);
                    return CompletedPart.builder()
                            .partNumber(finalI)
                            .eTag(uploadPartResponse.eTag())
                            .build();
                });
                futures.add(future);
            }
            List completedParts = new ArrayList<>();
            for (Future future : futures) {
                completedParts.add(future.get());
            }
            // 完成分段上传
            CompleteMultipartUploadRequest completeRequest = CompleteMultipartUploadRequest.builder()
                    .bucket(bucketName)
                    .key(key)
                    .uploadId(createResponse.uploadId())
                    .multipartUpload(CompletedMultipartUpload.builder()
                            .parts(completedParts)
                            .build())
                    .build();
            s3.completeMultipartUpload(completeRequest);
            System.out.println("Multipart upload completed.");
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            // 关闭 S3 客户端和线程池
            s3.close();
            executorService.shutdown();
        }
    }*/
    // 上传大文件
    /*public static String uploadFile(S3TransferManager transferManager, String bucketName,
                                    String baseDir, MultipartFile file, String catalog){
        // 生成新文件名
        String fileName = FileUploadUtils.extractFilename(file);
        String keyName = baseDir + catalog + "/" + fileName;
        log.info("生成的文件路径===》 {} ", keyName);
        try{
            UploadFileRequest uploadFileRequest =
                    UploadFileRequest.builder()
                            .putObjectRequest(b -> b.bucket(bucketName).key(keyName))
                            .addTransferListener(LoggingTransferListener.create())
                            //.source(file.)
                            .build();
            FileUpload fileUpload = transferManager.uploadFile(uploadFileRequest);
            CompletedFileUpload uploadResult = fileUpload.completionFuture().join();
            String s = uploadResult.response().eTag();
            return s;
        }catch (Exception e){
            log.info("======",e);
        }
        return "";
    }*/
    /**
     * 上传大文件
     * @param transferManager
     * @param bucketName
     * @param key
     * @param filePath
     * @return
     */
    /*public String uploadFile(S3TransferManager transferManager, String bucketName,
                             String key, String filePath) {
        UploadFileRequest uploadFileRequest =
                UploadFileRequest.builder()
                        .putObjectRequest(b -> b.bucket(bucketName).key(key))
                        .addTransferListener(LoggingTransferListener.create())
                        .source(Paths.get(filePath))
                        .build();
        FileUpload fileUpload = transferManager.uploadFile(uploadFileRequest);
        CompletedFileUpload uploadResult = fileUpload.completionFuture().join();
        return uploadResult.response().eTag();
    }*/
}

        国内使用s3的坑点: 

        亚马逊中国区，必须进行ICP 备案，申请开发80或443端口，否则是不能通过网址直接访问s3内部对象的。

相关推荐

关于我们常见问题广告服务免责声明联系我们
Powered By Z-BlogPHP 1.7.3 Theme By 优美主题
Copyright Your WebSite.Some Rights Reserved. 浙ICP备2023018783号