diff --git a/README.md b/README.md index 541add7..3d8ef0e 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,4 @@ +![Perplexed: An Obsidian Plugin for Perplexity and Perplexica](https://i.imgur.com/MVOK3rk.png) # Perplexed: AI Content Generation for Obsidian **Perplexed** is an Obsidian plugin that enables AI-powered content generation with source citations using [Perplexity](https://www.perplexity.ai/) and [Perplexica](https://perplexica.io/). This plugin brings research-grade AI capabilities directly into your Obsidian workspace, allowing you to generate well-cited content for your notes. @@ -5,6 +6,15 @@ # Perplexed: AI Content Generation for Obsidian ## 🎯 Key Features - **Source-Cited AI Responses**: Get AI-generated content with proper citations and references + - Default format: + > ```markdown + ### Citations + +[1]: 2024, Dec 13. [What is GRC (Governance, Risk and Compliance) - Metricstream](https://www.metricstream.com/learn/what-is-grc.html). Published: 2024-05-01 | Updated: 2024-12-13 + +[2]: 2025, Jun 16. [Governance, risk and compliance (GRC): Definitions and resources](https://www.diligent.com/resources/guides/grc). Published: 2025-05-27 | Updated: 2025-06-16 + > ``` + - **Multiple AI Providers**: Support for Perplexity (commercial) and Perplexica (self-hosted) - **Streaming Responses**: Real-time streaming of AI responses for better UX - **Flexible Configuration**: Customizable endpoints, models, and parameters diff --git a/src/docs/Understanding-the-Perplexity-Response-Object.md b/src/docs/Understanding-the-Perplexity-Response-Object.md new file mode 100644 index 0000000..a00ffd6 --- /dev/null +++ b/src/docs/Understanding-the-Perplexity-Response-Object.md @@ -0,0 +1,108 @@ +# Understanding the Perplexity Response Object + +Perplexity Streaming Response Data: +```json +{ + "id": "3cb1daed-3695-4248-8341-f37a7311e1fa", + "model": "sonar-pro", + "created": 1754570688, + "usage": { + "prompt_tokens": 322, + "completion_tokens": 814, + "total_tokens": 1136, + "search_context_size": "low", + "cost": { + "input_tokens_cost": 0.000966, + "output_tokens_cost": 0.01221, + "request_cost": 0.006, + "total_cost": 0.019176 + } + }, + "citations": [ + "https://www.metricstream.com/learn/what-is-grc.html", + "https://www.diligent.com/resources/guides/grc", + "https://www.ibm.com/think/topics/grc", + "https://www.bmc.com/blogs/grc-governance-risk-compliance/", + "https://www.salesforce.com/au/blog/what-is-grc/" + ], + "search_results": [ + { + "title": "What is GRC (Governance, Risk and Compliance) - Metricstream", + "url": "https://www.metricstream.com/learn/what-is-grc.html", + "date": "2024-05-01", + "last_updated": "2024-12-13" + }, + { + "title": "Governance, risk and compliance (GRC): Definitions and resources", + "url": "https://www.diligent.com/resources/guides/grc", + "date": "2025-05-27", + "last_updated": "2025-06-16" + }, + { + "title": "What Is GRC? | IBM", + "url": "https://www.ibm.com/think/topics/grc", + "date": "2021-10-08", + "last_updated": "2025-07-22" + }, + { + "title": "What Is GRC? Governance, Risk, and Compliance Explained", + "url": "https://www.bmc.com/blogs/grc-governance-risk-compliance/", + "date": "2024-12-24", + "last_updated": "2024-12-29" + }, + { + "title": "What is Governance Risk and Compliance Software (GRC)?", + "url": "https://www.salesforce.com/au/blog/what-is-grc/", + "date": "2024-08-29", + "last_updated": "2025-08-07" + } + ], + "images": [ + { + "image_url": "https://tallyfy.com/wp-content/uploads/2017/03/Guide-to-Governance-Risk-Management-and-Compliance.jpg", + "origin_url": "https://tallyfy.com/guides/governance-risk-management-compliance-grc/", + "height": 441, + "width": 753 + }, + { + "image_url": "https://d2908q01vomqb2.cloudfront.net/22d200f8670dbdb3e253a90eee5098477c95c23d/2018/11/30/Scaling-GRC-fullsize.jpg", + "origin_url": "https://aws.amazon.com/blogs/security/scaling-a-governance-risk-and-compliance-program-for-the-cloud/", + "height": 1688, + "width": 3000 + }, + { + "image_url": "https://appsiansecurity.com/wp-content/uploads/2021/12/GRC_diagram.png", + "origin_url": "https://appsiansecurity.com/governance-risk-compliance/", + "height": 1024, + "width": 1280 + }, + { + "image_url": "https://www.techtarget.com/rms/onlineImages/compliance-grc_framework-f_mobile.png", + "origin_url": "https://www.techtarget.com/searchsecurity/definition/governance-risk-management-and-compliance-GRC", + "height": 403, + "width": 560 + }, + { + "image_url": "https://s7280.pcdn.co/wp-content/uploads/2020/07/GRC-break-down.png", + "origin_url": "https://www.bmc.com/blogs/grc-governance-risk-compliance/", + "height": 469, + "width": 487 + } + ], + "object": "chat.completion", + "choices": [ + { + "index": 0, + "finish_reason": "stop", + "message": { + "role": "assistant", + "content": "**GRC Related Technology Applications: Empowering Modern Organizations**\n\n**Introduction**\n\nGovernance, Risk, and Compliance (GRC) related technology applications are digital tools and platforms designed to help organizations align their strategies, manage risks, and comply with an increasingly complex array of regulatory requirements[1][5]. As threatsβ€”ranging from cyber risks to regulatory changesβ€”grow more challenging, leveraging GRC technology has become essential for safeguarding operations, ensuring accountability, and enhancing decision-making.\n\n[IMAGE 1: GRC related technology applications concept diagram or illustration]\n\n**Main Content**\n\nGRC technology applications integrate governance policies, risk management practices, and compliance activities into unified platforms, enabling companies to move from siloed, manual processes to coordinated, automated systems[2][3]. This integration allows organizations to establish consistent policies, respond promptly to evolving regulations, and make informed decisions using real-time analytics and dashboards[2].\n\nPractical examples of GRC applications include software to:\n- Automate compliance tracking across industry standards (e.g., GDPR, SOX),\n- Manage audit workflows and store supporting documentation,\n- Monitor and assess security and operational risks,\n- Track and enforce policy distribution and acknowledgment within organizations[1][2][4].\n\nA typical use case is in the financial sector, where banks employ GRC platforms to ensure continual compliance with global anti-money laundering regulations, while automating the creation of audit trails and policy acknowledgments. In the healthcare industry, GRC tools help manage patient privacy risks and verify compliance with HIPAA rules.\n\nThe benefits of deploying GRC technologies are substantial. Automation reduces time and labor needed for repetitive compliance tasks, significantly cutting human error and compliance-related costs[2][4]. Integrated risk visibility and reporting ensure leaders can rapidly respond to emerging threats or regulatory changes. Customizable workflows let organizations adjust the software to unique processes, risk appetites, and industry-specific needs.\n\nHowever, organizations face several challenges when implementing GRC solutions. Technology alone cannot guarantee ethical conduct or drive organizational culture[4]. Effective deployment requires integrating technology with strong governance frameworks and processes, as well as adequate user training. Additionally, the diversity and complexity of regulatory environments can make initial implementation time-consuming, stressing the need for scalable and adaptable platforms[2][4].\n\n[IMAGE 2: GRC related technology applications practical example or use case]\n\n**Current State and Trends**\n\nGRC technology adoption is accelerating, driven by regulatory expansion, increasing risk exposure, and demands for greater operational visibility[5]. Leading enterprise platforms now offer integrated modules for risk, compliance, audit, policy, and vendor managementβ€”all within single, central interfaces[1][2]. Features such as real-time risk analytics, artificial intelligence-driven alerts, and regulatory intelligence updates are becoming standard in top solutions[2][3].\n\nKey industry players include MetricStream, Diligent, IBM, and Salesforce, each offering scalable, cloud-based GRC platforms[2][3][5]. Recent advances focus on automating document management, smart alerts based on AI, and tighter integration with third-party systems for vendor and IT risk management[2][3]. Markets are also seeing a rise in customizable workflows and self-service dashboards, which empower organizations of all sizes and complexities.\n\n[IMAGE 3: GRC related technology applications future trends or technology visualization]\n\n**Future Outlook**\n\nLooking ahead, GRC technology applications will likely leverage advanced analytics, machine learning, and expanded automation to proactively identify risks and ensure compliance in near real-time. As regulatory complexity and cyber threats intensify, seamless integration of GRC tools into core business operations will become a standard expectation. The continued evolution of cloud-based offerings, combined with AI-driven insights, promises to make GRC technology vital not just for risk management, but for organizational resilience and strategic innovation.\n\n**Conclusion**\n\nGRC related technology applications are transforming how organizations manage risk, compliance, and governance in an unpredictable world. As technology evolves, these solutions will be central to maintaining trust, transparency, and operational agility." + }, + "delta": { + "role": "assistant", + "content": "" + } + } + ] +} +``` \ No newline at end of file diff --git a/src/services/lmStudioService.ts b/src/services/lmStudioService.ts index 269e95f..bf14528 100644 --- a/src/services/lmStudioService.ts +++ b/src/services/lmStudioService.ts @@ -94,7 +94,15 @@ export class LMStudioService { if (this.settings.requestTemplate) { try { const processedTemplate = this.promptsService?.processTemplate(this.settings.requestTemplate) || this.settings.requestTemplate; - payload = JSON.parse(processedTemplate); + + // Strip JavaScript-style comments that would break JSON parsing + const cleanedTemplate = processedTemplate + .replace(/\/\*[\s\S]*?\*\//g, '') // Remove /* */ comments + .replace(/\/\/.*$/gm, '') // Remove // comments + .replace(/^\s*$/gm, '') // Remove empty lines + .trim(); + + payload = JSON.parse(cleanedTemplate); // Override with current parameters payload.model = model; payload.messages = messages; diff --git a/src/services/perplexicaService.ts b/src/services/perplexicaService.ts index 1a9b481..ce26bd2 100644 --- a/src/services/perplexicaService.ts +++ b/src/services/perplexicaService.ts @@ -78,7 +78,15 @@ export class PerplexicaService { if (this.settings.requestTemplate) { try { const processedTemplate = this.promptsService?.processTemplate(this.settings.requestTemplate) || this.settings.requestTemplate; - payload = JSON.parse(processedTemplate); + + // Strip JavaScript-style comments that would break JSON parsing + const cleanedTemplate = processedTemplate + .replace(/\/\*[\s\S]*?\*\//g, '') // Remove /* */ comments + .replace(/\/\/.*$/gm, '') // Remove // comments + .replace(/^\s*$/gm, '') // Remove empty lines + .trim(); + + payload = JSON.parse(cleanedTemplate); // Override with current parameters payload.chatModel = { provider: "ollama", diff --git a/src/services/perplexityService.ts b/src/services/perplexityService.ts index 91265ca..b109e9c 100644 --- a/src/services/perplexityService.ts +++ b/src/services/perplexityService.ts @@ -1,4 +1,5 @@ -import { Editor, Notice } from 'obsidian'; +import { Editor, Notice, request } from 'obsidian'; +import { formatCitationDate, getMostRecentDate, formatPublicationInfo } from '../utils/formatDate'; export interface PerplexityOptions { return_citations?: boolean; @@ -47,42 +48,82 @@ export class PerplexityService { } private processContentWithImages(content: string, images: any[]): string { + if (!images || images.length === 0) return content; + let processedContent = content; let imageIndex = 0; - - // Find image markers like [IMAGE 1: description] or [IMAGE 1: description] const imageRegex = /\[IMAGE\s+(\d+):\s*(.*?)\]/gi; let match; while ((match = imageRegex.exec(content)) !== null && imageIndex < images.length) { - const imageNumber = parseInt(match[1] || '0'); - const description = match[2]?.trim() || ''; + const [fullMatch, , description] = match; const image = images[imageIndex]; if (image && image.image_url) { - // Create the image markdown with description and source - let imageMarkdown = `\n\n![${description}](${image.image_url})\n`; - if (image.origin_url) { - imageMarkdown += `*Source: ${image.origin_url}*\n`; - } - imageMarkdown += '\n'; - - // Replace the marker with the actual image - processedContent = processedContent.replace(match[0], imageMarkdown); - console.log(`πŸ–ΌοΈ Replaced [IMAGE ${imageNumber}] with actual image: ${image.image_url}`); + const imageMarkdown = `![${description || 'Image'}](${image.image_url})`; + processedContent = processedContent.replace(fullMatch, imageMarkdown); } imageIndex++; } - // If we found and replaced any markers, return the processed content - if (imageIndex > 0) { - console.log(`πŸ”„ Processed ${imageIndex} image markers in content`); - return processedContent; - } + return processedContent; + } + + private addCitations(editor: Editor, sources: any[]): void { + if (!sources || sources.length === 0) return; + + console.log('πŸ“š Processing sources:', JSON.stringify(sources, null, 2)); + + // Add a section for citations + let citationsText = '\n\n### Citations\n\n'; - // If no markers were found, return original content - return content; + sources.forEach((source, index) => { + console.log(`πŸ“– Source ${index + 1}:`, source); + + // Handle different formats: + // 1. search_results format: {title, url, date, last_updated} + // 2. citations format: just URL strings + let title: string; + let url: string; + let formattedDate = ''; + let publicationInfo = ''; + + if (typeof source === 'string') { + // citations array format - just URLs + url = source; + title = 'Source'; + } else if (source && typeof source === 'object') { + // search_results format - detailed objects + title = source.title || 'Source'; + url = source.url || '#'; + + // Format the most recent date for the footnote + const mostRecentDate = getMostRecentDate(source); + if (mostRecentDate) { + formattedDate = formatCitationDate(mostRecentDate); + } + + // Create publication info string + publicationInfo = formatPublicationInfo(source); + } else { + // Fallback + title = 'Source'; + url = '#'; + } + + console.log(`πŸ“ Extracted - Title: "${title}", URL: "${url}", Date: "${formattedDate}", Info: "${publicationInfo}"`); + + // Format: [1]: 2024, Dec 13. [Title](URL). Published: date | Updated: date + citationsText += `[${index + 1}]: ${formattedDate ? formattedDate + '. ' : ''}[${title}](${url}).${publicationInfo ? ' ' + publicationInfo : ''}\n\n`; + }); + + console.log('πŸ“„ Final citations text:', citationsText); + + // Insert citations at the end of the editor content + const endOfDoc = editor.lastLine(); + const endPos = { line: endOfDoc, ch: editor.getLine(endOfDoc).length }; + editor.replaceRange(citationsText, endPos); } public async queryPerplexity( @@ -136,7 +177,38 @@ export class PerplexityService { if (this.settings.requestTemplate) { try { const processedTemplate = this.promptsService?.processTemplate(this.settings.requestTemplate) || this.settings.requestTemplate; - payload = JSON.parse(processedTemplate); + + // Strip JavaScript-style comments that would break JSON parsing + let cleanedTemplate = processedTemplate + .replace(/\/\*[\s\S]*?\*\//g, '') // Remove /* */ comments + .replace(/\/\/.*$/gm, '') // Remove // comments + .replace(/^\s*$/gm, '') // Remove empty lines + .trim(); + + // Check if template contains JavaScript code instead of JSON + if (cleanedTemplate.includes('const ') || cleanedTemplate.includes('fetch(') || cleanedTemplate.includes('await ')) { + console.warn('⚠️ Template contains JavaScript code, not JSON. Extracting payload object...'); + + // Try to extract JSON payload from JavaScript code + const payloadMatch = cleanedTemplate.match(/payload\s*=\s*({[\s\S]*?});/); + if (payloadMatch) { + let jsObject = payloadMatch[1]; + console.log('πŸ” Extracted payload from JavaScript:', jsObject); + + // Convert JavaScript object syntax to valid JSON + // Replace unquoted property names with quoted ones + jsObject = jsObject.replace(/(\w+):/g, '"$1":'); + // Fix single quotes to double quotes + jsObject = jsObject.replace(/'/g, '"'); + + cleanedTemplate = jsObject; + } else { + throw new Error('Template contains JavaScript code but no valid payload object found. Please use JSON format only.'); + } + } + + console.log('🧹 Final cleaned template:', cleanedTemplate); + payload = JSON.parse(cleanedTemplate); // Override with current query and options payload.model = model; payload.messages = [ @@ -177,44 +249,89 @@ export class PerplexityService { payload.search_recency_filter = convertedFilter; } + // Add cache busting to prevent request/response caching + const requestId = Date.now() + Math.random(); + // Debug: Log the API payload being sent - console.log('πŸš€ Perplexity API Payload:', JSON.stringify(payload, null, 2)); + console.log(`πŸš€ Perplexity API Payload [${requestId}]:`, JSON.stringify(payload, null, 2)); - const response = await fetch(this.settings.perplexityEndpoint, { - method: 'POST', - headers: { - 'Authorization': `Bearer ${this.settings.perplexityApiKey}`, - 'Content-Type': 'application/json' - }, - body: JSON.stringify(payload) - }); + try { + if (useStreaming) { + // Use fetch for streaming responses with cache busting headers + const response = await fetch(this.settings.perplexityEndpoint, { + method: 'POST', + headers: { + 'Authorization': `Bearer ${this.settings.perplexityApiKey}`, + 'Content-Type': 'application/json', + 'Accept': 'text/event-stream' + }, + body: JSON.stringify(payload), + cache: 'no-store' + }); - // Debug: Log the response status - console.log('πŸ“‘ Perplexity API Response Status:', response.status); - console.log('πŸ“‘ Perplexity API Response Headers:', { - 'content-type': response.headers.get('content-type'), - 'x-ratelimit-remaining': response.headers.get('x-ratelimit-remaining'), - 'x-ratelimit-reset': response.headers.get('x-ratelimit-reset') - }); + if (!response.ok) { + throw new Error(`HTTP error! status: ${response.status}`); + } - if (!response.ok) { - throw new Error(`HTTP error! status: ${response.status}`); - } - - let finalCursor = responseCursor; - - if (useStreaming) { - await this.handleStreamingResponse(response, editor, responseCursor); - } else { - await this.handleNonStreamingResponse(response, editor, responseCursor); - } - - // Add separator at the final cursor position - editor.replaceRange('\n\n***\n', finalCursor); - - // Close loading notice if it exists - if (loadingNotice) { - loadingNotice.hide(); + if (!response.body) { + throw new Error('No response body'); + } + + await this.handleStreamingResponse(response, editor, responseCursor, requestId); + } else { + // Use Obsidian's request method for non-streaming with cache busting + const response = await request({ + url: this.settings.perplexityEndpoint, + method: 'POST', + headers: { + 'Authorization': `Bearer ${this.settings.perplexityApiKey}`, + 'Content-Type': 'application/json', + 'Accept': 'application/json' + }, + body: JSON.stringify(payload) + }); + + // Parse the response + const data = JSON.parse(response); + + // Process the response + if (data.choices && data.choices.length > 0) { + let content = data.choices[0].message.content; + + // Process images if available + if (options?.return_images && data.images) { + content = this.processContentWithImages(content, data.images); + } + + // Insert the response at the cursor position + editor.replaceRange(content, responseCursor); + + // Add citations if available + if (options?.return_citations) { + if (data.search_results && data.search_results.length > 0) { + // Use search_results for detailed info (preferred for Perplexity) + this.addCitations(editor, data.search_results); + } else if (data.citations && data.citations.length > 0) { + // Fallback to citations array (could be URLs or other format) + this.addCitations(editor, data.citations); + } + } + } + + // Add separator at the final cursor position + editor.replaceRange('\n\n***\n', responseCursor); + } + + } catch (error) { + console.error('Error making request to Perplexity API:', error); + const errorMsg = error instanceof Error ? error.message : String(error); + new Notice(`Perplexity Error: ${errorMsg}`); + editor.replaceRange(`\n**Error:** ${errorMsg}\n\n***\n`, editor.getCursor()); + } finally { + // Close loading notice if it exists + if (loadingNotice) { + loadingNotice.hide(); + } } } catch (error) { @@ -232,146 +349,99 @@ export class PerplexityService { private async handleStreamingResponse( response: Response, editor: Editor, - responseCursor: { line: number; ch: number } + responseCursor: { line: number; ch: number }, + requestId?: number ): Promise { const reader = response.body?.getReader(); if (!reader) throw new Error('No response body'); + console.log(`πŸ”„ Starting streaming response handler [${requestId || 'unknown'}]`); + let buffer = ''; - let currentPos = responseCursor; + let currentPos = { ...responseCursor }; let finalResponseData: any = null; - while (true) { - const { done, value } = await reader.read(); - if (done) break; - - const chunk = new TextDecoder().decode(value); - buffer += chunk; - - // Debug: Log streaming chunks (verbose - uncomment if needed) - // console.log('πŸ“¦ Perplexity Streaming chunk received:', chunk); - - // Process complete lines from buffer - const lines = buffer.split('\n'); - buffer = lines.pop() || ''; // Keep incomplete line in buffer - - for (const line of lines) { - if (line.trim().startsWith('data: ')) { - const data = line.replace('data: ', '').trim(); - if (data === '[DONE]') continue; + try { + while (true) { + const { done, value } = await reader.read(); + if (done) break; + + const chunk = new TextDecoder().decode(value, { stream: true }); + buffer += chunk; + + // Process complete lines from buffer + const lines = buffer.split('\n'); + buffer = lines.pop() || ''; // Keep incomplete line in buffer + + for (const line of lines) { + if (line.trim() === '') continue; try { - const parsed = JSON.parse(data); - - // Store the final response data for processing citations and images - if (parsed.citations || parsed.images || parsed.search_results) { - finalResponseData = parsed; - } - - if (parsed.choices?.[0]?.delta?.content) { - const content = parsed.choices[0].delta.content; - editor.replaceRange(content, currentPos); - // Update cursor position after insertion - const lines = content.split('\n'); - if (lines.length === 1) { - currentPos = { line: currentPos.line, ch: currentPos.ch + content.length }; - } else { - currentPos = { - line: currentPos.line + lines.length - 1, - ch: lines[lines.length - 1]?.length || 0 - }; + if (line.startsWith('data: ')) { + const data = line.replace('data: ', '').trim(); + if (data === '[DONE]') continue; + + const parsed = JSON.parse(data); + + // Store the final response data for processing citations and images + // Always update finalResponseData to ensure we have the latest metadata + if (parsed.citations || parsed.images || parsed.search_results) { + finalResponseData = parsed; + } + + // Also clear any stale data if this chunk doesn't have metadata + // This prevents using old metadata from previous requests + if (parsed.choices?.[0]?.finish_reason === 'stop' && !parsed.citations && !parsed.images && !parsed.search_results) { + // This is the final chunk but has no metadata, ensure we don't use stale data + if (finalResponseData && !finalResponseData.choices?.[0]?.finish_reason) { + // Only clear if the stored data doesn't have finish_reason (meaning it's incomplete) + console.log('🧹 Clearing potentially stale finalResponseData'); + finalResponseData = null; + } + } + + if (parsed.choices?.[0]?.delta?.content) { + const content = parsed.choices[0].delta.content; + if (content) { + editor.replaceRange(content, currentPos); + // Update cursor position after insertion + const contentLines = content.split('\n'); + if (contentLines.length === 1) { + currentPos.ch += content.length; + } else { + currentPos.line += contentLines.length - 1; + currentPos.ch = contentLines[contentLines.length - 1].length; + } + // Scroll to follow the new content + editor.scrollIntoView({ from: currentPos, to: currentPos }, true); + } } - // Scroll to follow the new content - editor.scrollIntoView({ from: currentPos, to: currentPos }, true); - // Small delay to make scrolling smoother - await new Promise(resolve => setTimeout(resolve, 10)); } } catch (e) { - // Ignore JSON parse errors for partial chunks + console.warn('Error processing streaming chunk:', e); + // Continue processing other lines even if one fails } } - } - } - - // After streaming is complete, add citations and images if available - if (finalResponseData) { - await this.processStreamingMetadata(finalResponseData, editor); - } - } - - private async handleNonStreamingResponse( - response: Response, - editor: Editor, - responseCursor: { line: number; ch: number } - ): Promise { - const data = await response.json(); - const content = data.choices?.[0]?.message?.content || 'No response received'; - - // Extract citations/sources if available - let fullResponse = content; - console.log('πŸ” Perplexity Response Data:', JSON.stringify(data, null, 2)); - - // Process images with intelligent placement - if (data.images && data.images.length > 0) { - console.log('πŸ“Έ Perplexity Images Found:', data.images.length); - console.log('πŸ“Έ Images Data:', JSON.stringify(data.images, null, 2)); - - // Process the content for image markers - const processedContent = this.processContentWithImages(content, data.images); - - if (processedContent !== content) { - // Use the processed content with inline images - fullResponse = processedContent; - console.log('πŸ”„ Content updated with inline images (non-streaming)'); - } else { - // Fallback: add images at the end if no markers found - fullResponse += '\n\n## Images\n\n'; - data.images.forEach((image: any, index: number) => { - if (image.image_url) { - fullResponse += `![Image ${index + 1}](${image.image_url})\n`; - if (image.origin_url) { - fullResponse += `*Source: ${image.origin_url}*\n\n`; - } else { - fullResponse += '\n'; - } - } - }); - } - } else { - console.log('πŸ“Έ No images found in Perplexity response'); - } - - editor.replaceRange(fullResponse, responseCursor); - - // Add sources after the last non-empty line if available - if (data.citations && data.citations.length > 0) { - console.log('πŸ“š Perplexity Sources Found:', data.citations.length); - console.log('πŸ“š Sources Data:', JSON.stringify(data.citations, null, 2)); - - // Find the last non-empty line - let lastNonEmptyLine = editor.lastLine(); - while (lastNonEmptyLine >= 0) { - const lineContent = editor.getLine(lastNonEmptyLine); - if (lineContent.trim() !== '') { - break; - } - lastNonEmptyLine--; - } - - // If we found a non-empty line, insert sources after it - if (lastNonEmptyLine >= 0) { - const lastLineContent = editor.getLine(lastNonEmptyLine); - const insertPosition = { line: lastNonEmptyLine, ch: lastLineContent.length }; - let sourcesSection = '\n\n## Sources\n\n'; - data.citations.forEach((citation: any, index: number) => { - sourcesSection += `[${index + 1}] ${citation.url || citation.title || citation}\n`; - }); - - editor.replaceRange(sourcesSection, insertPosition); + // Small delay to prevent UI blocking + await new Promise(resolve => setTimeout(resolve, 10)); } - } else { - console.log('πŸ“š No sources/citations found in Perplexity response'); + + // Process final metadata (citations, images) after streaming is complete + if (finalResponseData) { + console.log(`πŸ“ Processing final response data [${requestId || 'unknown'}]:`, finalResponseData); + await this.processStreamingMetadata(finalResponseData, editor); + } + + // Add final separator + const endPos = { ...currentPos }; + editor.replaceRange('\n\n***\n', endPos); + + } catch (error) { + console.error('Error in streaming response:', error); + const errorMsg = error instanceof Error ? error.message : 'Unknown error during streaming'; + new Notice(`Streaming error: ${errorMsg}`); + editor.replaceRange(`\n**Streaming Error:** ${errorMsg}\n\n`, currentPos); } } @@ -383,17 +453,13 @@ export class PerplexityService { // Process images with intelligent placement if (finalResponseData.images && finalResponseData.images.length > 0) { - console.log('πŸ“Έ Perplexity Images Found (Streaming):', finalResponseData.images.length); - console.log('πŸ“Έ Images Data (Streaming):', JSON.stringify(finalResponseData.images, null, 2)); + const content = editor.getValue(); + const processedContent = this.processContentWithImages(content, finalResponseData.images); - // Get the current content to process for image markers - const currentContent = editor.getValue(); - const processedContent = this.processContentWithImages(currentContent, finalResponseData.images); - - if (processedContent !== currentContent) { - // Replace the entire content with processed version + if (processedContent !== content) { + // Update the editor with the processed content editor.setValue(processedContent); - console.log('πŸ”„ Content updated with inline images'); + console.log('πŸ”„ Content updated with inline images (streaming)'); } else { // Fallback: add images at the end if no markers found let imagesSection = '\n\n## Images\n\n'; @@ -408,41 +474,22 @@ export class PerplexityService { } }); - editor.replaceRange(imagesSection, editor.getCursor()); + // Append images section to the end of the document + const endOfDoc = editor.lastLine(); + const endPos = { line: endOfDoc, ch: editor.getLine(endOfDoc).length }; + editor.replaceRange(imagesSection, endPos); } - } else { - console.log('πŸ“Έ No images found in Perplexity streaming response'); } - // Add citations/sources after the last non-empty line if available - if (finalResponseData.citations && finalResponseData.citations.length > 0) { - console.log('πŸ“š Perplexity Sources Found (Streaming):', finalResponseData.citations.length); - console.log('πŸ“š Sources Data (Streaming):', JSON.stringify(finalResponseData.citations, null, 2)); - - // Find the last non-empty line - let lastNonEmptyLine = editor.lastLine(); - while (lastNonEmptyLine >= 0) { - const lineContent = editor.getLine(lastNonEmptyLine); - if (lineContent.trim() !== '') { - break; - } - lastNonEmptyLine--; - } - - // If we found a non-empty line, insert sources after it - if (lastNonEmptyLine >= 0) { - const lastLineContent = editor.getLine(lastNonEmptyLine); - const insertPosition = { line: lastNonEmptyLine, ch: lastLineContent.length }; - - let sourcesSection = '\n\n## Sources\n\n'; - finalResponseData.citations.forEach((citation: any, index: number) => { - sourcesSection += `[${index + 1}] ${citation}\n`; - }); - - editor.replaceRange(sourcesSection, insertPosition); - } + // Process sources/citations if available + if (finalResponseData.search_results && finalResponseData.search_results.length > 0) { + // Use search_results for detailed info (preferred for Perplexity streaming) + this.addCitations(editor, finalResponseData.search_results); + } else if (finalResponseData.citations && finalResponseData.citations.length > 0) { + // Fallback to citations array (could be URLs or other format) + this.addCitations(editor, finalResponseData.citations); } else { console.log('πŸ“š No sources/citations found in Perplexity streaming response'); } } -} \ No newline at end of file +} \ No newline at end of file diff --git a/src/utils/formatDate.ts b/src/utils/formatDate.ts new file mode 100644 index 0000000..cae7701 --- /dev/null +++ b/src/utils/formatDate.ts @@ -0,0 +1,69 @@ +/** + * Date formatting utilities for the Perplexed plugin + */ + +/** + * Formats a date string into the format: "YYYY, MMM DD" + * Examples: "2024, Dec 13", "2025, Jan 05" + * + * @param dateString - Date string in various formats (ISO, YYYY-MM-DD, etc.) + * @returns Formatted date string or empty string if invalid + */ +export function formatCitationDate(dateString: string): string { + if (!dateString) return ''; + + try { + const date = new Date(dateString); + + // Check if date is valid + if (isNaN(date.getTime())) { + return ''; + } + + const year = date.getFullYear(); + const month = date.toLocaleDateString('en-US', { month: 'short' }); + const day = date.getDate().toString().padStart(2, '0'); + + return `${year}, ${month} ${day}`; + } catch (error) { + console.warn('Failed to format date:', dateString, error); + return ''; + } +} + +/** + * Gets the most recent date from available date fields + * Prioritizes last_updated over date + * + * @param source - Source object with potential date fields + * @returns Most recent date string or empty string if none available + */ +export function getMostRecentDate(source: any): string { + if (source.last_updated) { + return source.last_updated; + } + if (source.date) { + return source.date; + } + return ''; +} + +/** + * Creates the publication info string for citations + * Format: "Published: YYYY-MM-DD | Updated: YYYY-MM-DD" + * + * @param source - Source object with potential date fields + * @returns Publication info string or empty string if no dates available + */ +export function formatPublicationInfo(source: any): string { + const dateInfo = []; + + if (source.date) { + dateInfo.push(`Published: ${source.date}`); + } + if (source.last_updated) { + dateInfo.push(`Updated: ${source.last_updated}`); + } + + return dateInfo.join(' | '); +} diff --git a/styles.css b/styles.css index ac3ecf5..4eb7e07 100644 --- a/styles.css +++ b/styles.css @@ -1 +1,72 @@ -.perplexity-modal .text-input{width:100%;margin:8px 0;padding:12px}.perplexity-modal .setting-item-description{font-size:12px;color:var(--text-muted);margin-top:5px}.perplexity-modal .setting-item-description.images-description{font-size:11px;margin-top:3px}.article-generator-modal .text-input{width:100%;margin:8px 0;padding:12px}.article-generator-modal .setting-item-description{font-size:12px;color:var(--text-muted);margin-top:5px}.article-generator-modal .term-description{display:block!important;margin-top:8px;margin-bottom:10px;width:100%!important;flex-basis:100%!important;order:2}.article-generator-modal .setting-item{flex-direction:column!important;align-items:flex-start!important}.article-generator-modal .setting-item>*{width:100%!important}.article-generator-modal .hidden-input{display:none}.perplexica-modal .text-input{width:100%;margin:8px 0;padding:12px}.lmstudio-modal .text-input{width:100%;margin:8px 0;padding:12px}.lmstudio-modal .system-prompt-input{width:100%;min-height:60px}.url-update-modal .text-input{width:100%;margin:8px 0;padding:12px} +/* src/styles/perplexity-modal.css */ +.perplexity-modal .text-input { + width: 100%; + margin: 8px 0; + padding: 12px; +} +.perplexity-modal .setting-item-description { + font-size: 12px; + color: var(--text-muted); + margin-top: 5px; +} +.perplexity-modal .setting-item-description.images-description { + font-size: 11px; + margin-top: 3px; +} + +/* src/styles/article-generator-modal.css */ +.article-generator-modal .text-input { + width: 100%; + margin: 8px 0; + padding: 12px; +} +.article-generator-modal .setting-item-description { + font-size: 12px; + color: var(--text-muted); + margin-top: 5px; +} +.article-generator-modal .term-description { + display: block !important; + margin-top: 8px; + margin-bottom: 10px; + width: 100% !important; + flex-basis: 100% !important; + order: 2; +} +.article-generator-modal .setting-item { + flex-direction: column !important; + align-items: flex-start !important; +} +.article-generator-modal .setting-item > * { + width: 100% !important; +} +.article-generator-modal .hidden-input { + display: none; +} + +/* src/styles/perplexica-modal.css */ +.perplexica-modal .text-input { + width: 100%; + margin: 8px 0; + padding: 12px; +} + +/* src/styles/lmstudio-modal.css */ +.lmstudio-modal .text-input { + width: 100%; + margin: 8px 0; + padding: 12px; +} +.lmstudio-modal .system-prompt-input { + width: 100%; + min-height: 60px; +} + +/* src/styles/url-update-modal.css */ +.url-update-modal .text-input { + width: 100%; + margin: 8px 0; + padding: 12px; +} + +/* src/styles/main.css */ diff --git a/test-perplexity-api.sh b/test-perplexity-api.sh new file mode 100755 index 0000000..787eafa --- /dev/null +++ b/test-perplexity-api.sh @@ -0,0 +1,39 @@ +#!/bin/bash + +# Test script for Perplexity API +# Usage: ./test-perplexity-api.sh YOUR_API_KEY + +if [ -z "$1" ]; then + echo "Usage: $0 YOUR_API_KEY" + echo "Example: $0 pplx-abc123..." + exit 1 +fi + +API_KEY="$1" + +echo "Testing Perplexity API with the same payload structure..." +echo "API Key: ${API_KEY:0:10}..." +echo "" + +curl -X POST "https://api.perplexity.ai/chat/completions" \ + -H "Authorization: Bearer $API_KEY" \ + -H "Content-Type: application/json" \ + -H "Accept: application/json" \ + -d '{ + "model": "sonar-pro", + "messages": [ + { + "role": "user", + "content": "In GRC platforms, what does Governance mean? Please explain important workflows, tools, processes, and any policies or regulations that might be relevant. If you mention tools or services or regulatory bodies or membership organizations, please link to them in the text.\n\n**Image References:**\nPlease include the following image references throughout your response where appropriate:\n- [IMAGE 1: Relevant diagram or illustration related to the topic]\n- [IMAGE 2: Practical example or use case visualization]\n- [IMAGE 3: Additional supporting visual content]" + } + ], + "stream": true, + "return_citations": true, + "return_images": true, + "return_related_questions": false + }' \ + --verbose \ + --max-time 30 + +echo "" +echo "Test completed." diff --git a/test-perplexity-non-streaming.sh b/test-perplexity-non-streaming.sh new file mode 100755 index 0000000..22988d6 --- /dev/null +++ b/test-perplexity-non-streaming.sh @@ -0,0 +1,39 @@ +#!/bin/bash + +# Test script for Perplexity API (Non-streaming) +# Usage: ./test-perplexity-non-streaming.sh YOUR_API_KEY + +if [ -z "$1" ]; then + echo "Usage: $0 YOUR_API_KEY" + echo "Example: $0 pplx-abc123..." + exit 1 +fi + +API_KEY="$1" + +echo "Testing Perplexity API (NON-STREAMING) with the same payload structure..." +echo "API Key: ${API_KEY:0:10}..." +echo "" + +curl -X POST "https://api.perplexity.ai/chat/completions" \ + -H "Authorization: Bearer $API_KEY" \ + -H "Content-Type: application/json" \ + -H "Accept: application/json" \ + -d '{ + "model": "sonar-pro", + "messages": [ + { + "role": "user", + "content": "What is GRC in simple terms?" + } + ], + "stream": false, + "return_citations": true, + "return_images": false, + "return_related_questions": false + }' \ + --verbose \ + --max-time 30 + +echo "" +echo "Non-streaming test completed."