Issues discussed with dify_helper

Feedback
,
1

Communicate with the Official Team: This is a Typical “Plugin Layer Error”

Based on the logs you provided, this is an issue that can and should be reported to the official team/plugin maintainers. We recommend you compile and submit the following information:

  1. Environment Information:

    • Dify Version: 1.11.4

    • Deployment Method: Docker / Self-hosted

    • Jina API Key is configured and the frontend shows “connected”

  2. Reproduction Steps (the simpler, the better):

    • Use the official “Knowledge Base Pipeline” template;

    • Select Jina Reader in the website data source node;

    • Enter any URL (e.g., https://example.com);

    • An error occurs after running.

  3. Error Message (the segment you just pasted is crucial and can be included as is):

    Error in stream response for plugin {'code': -500, 'message': '{\"message\":\"{\\\\\"args\\\\\":{},\\\\\"error_type\\\\\":\\\\\"ValueError\\\\\",\\\\\"message\\\\\":\\\\\"An error occurred: Failed to crawl\\\\\"}\",\"error_type\":\"PluginInvokeError\",\"args\":null}', 'data': None}\n   Error during website crawl.\n   Traceback (most recent call last):\n     File \"/app/api/services/rag_pipeline/rag_pipeline.py\", line 631, in run_datasource_workflow_node\n       for website_crawl_message in website_crawl_result:\n     File \"/app/api/core/plugin/impl/base.py\", line 306, in _request_with_plugin_daemon_response_stream\n       self._handle_plugin_daemon_error(error.error_type, error.message)\n     File \"/app/api/core/plugin/impl/base.py\", line 348, in _handle_plugin_daemon_error\n       raise PluginInvokeError(description=message)\n   core.plugin.impl.exc.PluginInvokeError: req_id: 5424c10501 PluginInvokeError: {\"args\":{},\"error_type\":\"ValueError\",\"message\":\"An error occurred: Failed to crawl\"}\n

  4. Additional Notes:

    • Inside the container, curl tests to the external network and jina.ai both return 200;

    • Other network-related functionalities work normally in the same environment.

This information should be sufficient for maintainers to determine:

  • Whether there’s a bug on the Jina Web Reader plugin side (plugin daemon / the layer that calls Jina);

  • Or if Jina has recently changed its API/permissions, causing the crawling logic to consistently throw Failed to crawl.