Fix for audio format chunking

README.md

@@ -18,7 +18,9 @@ Getting started with the Smart Memos Plugin is easy. Follow these steps to insta
 1. Download and install the Smart Memos Plugin from the Obsidian Community Plugins.
 2. Configure the plugin settings with your OpenAI API Key and preferred AI model.
 
-Note* This plugin currently uses online openAI models to recieve and transcribe your voice memos.  Looking to add Local AI Models in the near future. OpenAI's retention policy retains text up to 30 days, but does not retain any audio data. 
+**Note** This plugin currently uses online openAI models to recieve and transcribe your voice memos.  Looking to add Local AI Models in the near future. OpenAI's retention policy retains text up to 30 days, but does not retain any audio data. 
+
+**Another Note:** If you have the native record feature in Obsidian turned on, it must be turned off for audio to record using Smart Memos.
 
 ## Platforms
 
@@ -28,12 +30,15 @@ Note* This plugin currently uses online openAI models to recieve and transcribe
 ## Usage
 Once installed, the Smart Memos Plugin provides an intuitive interface to transcribe your audio files and generate notes.  
 
-- **Adding Audio**: Ensure you have a note opened. To speak your memo directly into Obsidian, tap the microphone icon or use the command `Record smart memo` from the command pallet (`Ctrl + p` for Windows and `Cmd + p` for Mac) to open the smart memos popup that'll automatically start recording.  To import audio into a note, simply drag and drop the file into it.
+- **Adding Audio**: To speak your memo directly into Obsidian, tap the microphone icon that displays "Record smart memo" or select `Record smart memo` from the command palette (`Ctrl + p` for Windows and `Cmd + p` for Mac) to open the smart memos popup that'll automatically start recording.  To import audio into a note, simply drag and drop an audio file into it.
 - **Transcribing Audio**: To transcribe an audio file after it's been imported into a note, move your cursor right underneath the audio file and use the command `Smart transcribe` from the command palette. The plugin will transcribe the audio file and generate detailed notes. If you're speaking directly to Obsidian, you can select the "Smart Transcribe" button to transcribe what you've recorded.
 - **Customizing the Prompt**: You can customize the prompt that will be sent to the AI model before adding your transcribed audio in the plugin settings.
 - **Include Raw Transcript**: If you just want clean notes returned, you can remove the addition of the raw transcript at the end by toggling it off in the settings
+- **Specify where audio files are recorded in your vault**: By default, audio recordings will be saved to your root vault folder.  If you want to store audio recordings in a specific folder, you can change it in the settings of this plugin. I.e if you want them to be saved in a 'Recordings' folder within your 'Resources' folder, you can set the settings value to Resources/Recordings.
 
-Note* This plugin finds your audio file by looking at what you have set at the "Default location for new attachments" within the "Files and Links" tab of Obsidian settings.  If your audio file is for some reason not stored at the location specified, you should move it there.  I will see about a workaround for missplaced audio files.
+## Coming Soon(ish)!
+- **Smart Templates**: Given the seemingly infinite use cases, I'm working with Brian (creator of smart connections) to integrate "Smart Templates", a templating feature that will be available in Obsidian within the coming weeks.
+- **Local Model Transcription**:  This is a high priority since it's understandable not as many are as comfortable having their voice audio sent out online to be transcribed. 
 
 ## Vision
 The Smart Memos plugin aims to revolutionize the way we capture and understand information from audio sources. By leveraging advanced AI models, the plugin can transcribe audio files and generate fully customizable notes, in-depth analysis, and idea expansion, freeing you from the tedious task of manual transcription and note-taking, while simultaneously expanding upon them to allow your two "brains" to work harmoneously.

SmartMemosAudioRecordModal.ts

@@ -5,7 +5,7 @@ export class SmartMemosAudioRecordModal extends Modal {
     private chunks: BlobPart[] = [];
     private resolve: (value: Blob | PromiseLike<Blob>) => void;
     private reject: (reason?: any) => void;
-    private handleAudioRecording: (audioFile: Blob | null, transcribe: boolean) => void;
+    private handleAudioRecording: (audioFile: Blob | null, transcribe: boolean, keepAudio: boolean, includeAudioFileLink: boolean) => void;
     private isRecording: boolean = false;
     private timer: HTMLElement;
     private intervalId: number | null = null;
@@ -13,10 +13,14 @@ export class SmartMemosAudioRecordModal extends Modal {
     private elapsedTime: number = 0; // To keep track of the elapsed time
     private redDot: HTMLElement;
     private isResetting: boolean = false; // Flag to track reset state
+    private keepAudioCheckbox: HTMLElement; // Add a property for the checkbox
+    private includeAudioFileLinkCheckbox: HTMLElement; // Add a property for the checkbox
+    private settings: any;
 
-    constructor(app: any, handleAudioRecording: (audioFile: Blob | null, transcribe: boolean) => void) {
+    constructor(app: any, handleAudioRecording: (audioFile: Blob | null, transcribe: boolean, keepAudio: boolean, includeAudioFileLink: boolean) => void, settings: any) {
         super(app);
         this.handleAudioRecording = handleAudioRecording;
+        this.settings = settings; // Initialize settings
     }
 
     onOpen() {
@@ -53,7 +57,7 @@ export class SmartMemosAudioRecordModal extends Modal {
 
         stopButton.addEventListener('click', async () => {
             const audioFile = await this.stopRecording();
-            this.handleAudioRecording(audioFile, false);
+            this.handleAudioRecording(audioFile, false, (this.keepAudioCheckbox as HTMLInputElement).checked, (this.includeAudioFileLinkCheckbox as HTMLInputElement).checked);
         });
 
         playPauseButton.addEventListener('click', () => {
@@ -77,7 +81,7 @@ export class SmartMemosAudioRecordModal extends Modal {
 
         transcribeButton.addEventListener('click', async () => {
             const audioFile = await this.stopRecording();
-            this.handleAudioRecording(audioFile, true);
+            this.handleAudioRecording(audioFile, true, (this.keepAudioCheckbox as HTMLInputElement).checked, (this.includeAudioFileLinkCheckbox as HTMLInputElement).checked);
         });
 
         setIcon(transcribeButton, 'file-text'); // Initially set to bulb
@@ -103,6 +107,19 @@ export class SmartMemosAudioRecordModal extends Modal {
             // Ensure red dot stops pulsing
             this.redDot.classList.remove('smart-memo-pulse-animation');
         });
+        // Add the checkbox
+        const keepAudioContainer = contentEl.createDiv({ cls: 'smart-memo-keep-audio-container' });
+        this.keepAudioCheckbox = keepAudioContainer.createEl('input', { type: 'checkbox', cls: 'smart-memo-keep-audio-checkbox' });
+        (this.keepAudioCheckbox as HTMLInputElement).checked = this.settings.keepAudio // Set checked based on settings;
+        const keepAudioLabel = keepAudioContainer.createEl('label', { text: 'Keep Audio File', cls: 'smart-memo-keep-audio-label' });
+        keepAudioLabel.htmlFor = this.keepAudioCheckbox.id;
+
+         // Add the checkbox for including audio file link
+         const includeAudioFileLinkContainer = contentEl.createDiv({ cls: 'smart-memo-include-audio-file-link-container' });
+         this.includeAudioFileLinkCheckbox = includeAudioFileLinkContainer.createEl('input', { type: 'checkbox', cls: 'smart-memo-include-audio-file-link-checkbox' });
+         (this.includeAudioFileLinkCheckbox as HTMLInputElement).checked = this.settings.includeAudioFileLink; // Set checked based on settings
+         const includeAudioFileLinkLabel = includeAudioFileLinkContainer.createEl('label', { text: 'Include Audio File Player', cls: 'smart-memo-include-audio-file-link-label' });
+         includeAudioFileLinkLabel.htmlFor = this.includeAudioFileLinkCheckbox.id;
 
         // Start recording immediately upon opening the modal
         this.startRecording();

Utils.ts

@@ -2,18 +2,8 @@ import { App, TFile, TFolder, normalizePath } from 'obsidian';
 
 export async function saveFile(app: App, audioBlob: Blob, fileName: string, path: string): Promise<TFile> {
     try {
-        
-        let normalizedPath: string;
-        try {
-            normalizedPath = normalizePath(path);
-            if (!normalizedPath) {
-                console.warn('Normalized path is invalid, using root path');
-                normalizedPath = '/'; // Set to root path of the vault
-            }
-        } catch (error) {
-            console.warn('Error normalizing path, using root path:', error);
-            normalizedPath = '/'; // Set to root path of the vault
-        }        const filePath = `${normalizedPath}/${fileName}`;
+        const normalizedPath = normalizePath(path);
+        const filePath = `${normalizedPath}/${fileName}`;
 
         await ensureDirectoryExists(app, normalizedPath);
 

main.ts

@@ -10,13 +10,19 @@ interface AudioPluginSettings {
     apiKey: string;
 	prompt: string;
     includeTranscript: boolean;
+    recordingFilePath: string;
+    keepAudio: boolean;
+    includeAudioFileLink : boolean;
 }
 
 let DEFAULT_SETTINGS: AudioPluginSettings = {
 	model: 'gpt-4-0613',
     apiKey: '',
 	prompt: 'You are an expert note-making AI for obsidian who specializes in the Linking Your Thinking (LYK) strategy.  The following is a transcription of recording of someone talking aloud or people in a conversation. There may be a lot of random things said given fluidity of conversation or thought process and the microphone\'s ability to pick up all audio.  Give me detailed notes in markdown language on what was said in the most easy-to-understand, detailed, and conceptual format.  Include any helpful information that can conceptualize the notes further or enhance the ideas, and then summarize what was said.  Do not mention \"the speaker\" anywhere in your response.  The notes your write should be written as if I were writting them. Finally, ensure to end with code for a mermaid chart that shows an enlightening concept map combining both the transcription and the information you added to it.  The following is the transcribed audio:\n\n',
-    includeTranscript: true
+    includeTranscript: true,
+    recordingFilePath: '',
+    keepAudio: true,
+    includeAudioFileLink: false
 }
 
 const MODELS: string[] = [
@@ -28,7 +34,8 @@ const MODELS: string[] = [
 	'code-davinci-001',
 	'gpt-4-0613',
 	'gpt-4-32k-0613',
-	'gpt-4o'
+	'gpt-4o',
+    'gpt-4o-mini'
 ];
   
 
@@ -36,12 +43,14 @@ export default class SmartMemosPlugin extends Plugin {
 	settings: AudioPluginSettings;
 	writing: boolean;
 	transcript: string;
-
 	apiKey: string = 'sk-as123mkqwenjasdasdj12...';
     model: string = 'gpt-4-0613';
 
     appJsonObj : any;
 
+    private audioContext: AudioContext;
+
+
     // Add a new property to store the audio file
     audioFile: Blob;
 
@@ -52,6 +61,8 @@ export default class SmartMemosPlugin extends Plugin {
         const app_json = await this.app.vault.adapter.read(".obsidian/app.json");
         this.appJsonObj = JSON.parse(app_json);
 
+        this.audioContext = new (window.AudioContext || (window as any).webkitAudioContext)();
+
 
 		this.addCommand({
 			id: 'open-transcript-modal',
@@ -61,23 +72,21 @@ export default class SmartMemosPlugin extends Plugin {
             }
 		});
 
-        // Add the record smart memo command
         this.addCommand({
             id: 'record-smart-memo',
             name: 'Record smart memo',
             editorCallback: async (editor: Editor, view: MarkdownView) => {
                 // Open the audio recorder and store the recorded audio
-                this.audioFile = await new SmartMemosAudioRecordModal(this.app, this.handleAudioRecording.bind(this)).open();
+                this.audioFile = await new SmartMemosAudioRecordModal(this.app, this.handleAudioRecording.bind(this), this.settings).open();
+
             }
         });
 
+
         this.registerMarkdownPostProcessor((el: HTMLElement, ctx: MarkdownPostProcessorContext) => {
             const audioLinks = el.querySelectorAll('a.internal-link[data-href$=".wav"]');
-            console.log('audio links: ', audioLinks);
             audioLinks.forEach(link => {
 
-                console.log('linksss');
-
                 const href = link.getAttribute('data-href');
                 if (href === null) {
                     console.error('Failed to get the href attribute from the link element.');
@@ -107,9 +116,10 @@ export default class SmartMemosPlugin extends Plugin {
 
 
          // Add the audio recorder ribbon
+         // Update the callback for the audio recorder ribbon
         this.addRibbonIcon('microphone', 'Record smart memo', async (evt: MouseEvent) => {
             // Open the audio recorder and store the recorded audio
-            this.audioFile = await new SmartMemosAudioRecordModal(this.app, this.handleAudioRecording.bind(this)).open();
+            this.audioFile = await new SmartMemosAudioRecordModal(this.app, this.handleAudioRecording.bind(this), this.settings).open();
 
         });
 
@@ -118,7 +128,7 @@ export default class SmartMemosPlugin extends Plugin {
 	}
 
     // Add a new method to handle the audio recording and processing
-    async handleAudioRecording(audioFile: Blob, transcribe: boolean) {
+    async handleAudioRecording(audioFile: Blob, transcribe: boolean, keepAudio: boolean, includeAudioFileLink: boolean) {
         try {
             console.log('Handling audio recording:', audioFile);
 
@@ -131,18 +141,25 @@ export default class SmartMemosPlugin extends Plugin {
 
             // Save the audio recording as a .wav file
             const fileName = `recording-${Date.now()}.wav`;
-            const file = await saveFile(this.app, this.audioFile, fileName, this.appJsonObj.attachmentFolderPath);
+            const file = await saveFile(this.app, this.audioFile, fileName, this.settings.recordingFilePath);
 
-            // Insert a link to the audio file in the current note
-            const activeView = this.app.workspace.getActiveViewOfType(MarkdownView);
-            if (activeView) {
-                const editor = activeView.editor;
-                const cursor = editor.getCursor();
-                const link = `![[${file.path}]]`;
-                editor.replaceRange(link, cursor);
+            this.settings.keepAudio = keepAudio;
+            this.settings.includeAudioFileLink = includeAudioFileLink;
+            this.saveSettings();
 
-                // Trigger a change in the editor to force Obsidian to re-render the note
-                editor.replaceRange('', { line: cursor.line, ch: cursor.ch }, { line: cursor.line, ch: cursor.ch });
+            // Only save the audio file if use wants to include it and they are keeping the audio
+            if (includeAudioFileLink && keepAudio) { 
+                // Insert a link to the audio file in the current note
+                const activeView = this.app.workspace.getActiveViewOfType(MarkdownView);
+                if (activeView) {
+                    const editor = activeView.editor;
+                    const cursor = editor.getCursor();
+                    const link = `![[${file.path}]]`;
+                    editor.replaceRange(link, cursor);
+    
+                    // Trigger a change in the editor to force Obsidian to re-render the note
+                    editor.replaceRange('', { line: cursor.line, ch: cursor.ch }, { line: cursor.line, ch: cursor.ch });
+                }
             }
 
             // Transcribe the audio file if the transcribe parameter is true
@@ -150,9 +167,6 @@ export default class SmartMemosPlugin extends Plugin {
                 this.transcribeRecording(file);
             }
 
-            // Handle the saved audio file
-            // You can replace this with your own handling logic
-            console.log(file);
         } catch (error) {
             console.error('Error handling audio recording:', error);
             new Notice('Failed to handle audio recording');
@@ -181,6 +195,10 @@ export default class SmartMemosPlugin extends Plugin {
                 const prompt = this.settings.prompt + result;
                 new Notice('Transcript generated...');
                 this.generateText(prompt, editor , editor.getCursor('to').line);
+                //if keepAudio is false and delete the audio file if so
+                if (!this.settings.keepAudio) {
+                    this.app.vault.delete(audioFile); // Delete the audio file
+                }
             }).catch(error => {
                 console.warn(error.message);
                 new Notice(error.message);
@@ -207,8 +225,10 @@ export default class SmartMemosPlugin extends Plugin {
 	commandGenerateTranscript(editor: Editor) {
         const position = editor.getCursor();
         const text = editor.getRange({ line: 0, ch: 0 }, position);
-        const regex = [/(?<=\[\[)(([^[\]])+)\.(mp3|mp4|mpeg|mpga|m4a|wav|webm)(?=]])/g,
-            /(?<=\[(.*)]\()(([^[\]])+)\.(mp3|mp4|mpeg|mpga|m4a|wav|webm)(?=\))/g];
+        const regex = [
+            /(?<=\[\[)(([^[\]])+)\.(mp3|mp4|mpeg|mpga|m4a|wav|webm)(?=]])/g,
+            /(?<=\[(.*)]\()(([^[\]])+)\.(mp3|mp4|mpeg|mpga|m4a|wav|webm)(?=\))/g
+        ];
         this.findFilePath(text, regex).then((path) => {
             const fileType = path.split('.').pop();
             if (fileType == undefined || fileType == null || fileType == '') {
@@ -227,7 +247,7 @@ export default class SmartMemosPlugin extends Plugin {
                             this.transcript = result;
                             const prompt = this.settings.prompt + result;
                             new Notice('Transcript generated...');
-                    this.generateText(prompt, editor , editor.getCursor('to').line);
+                            this.generateText(prompt, editor, editor.getCursor('to').line);
                         }).catch(error => {
                             console.warn(error.message);
                             new Notice(error.message);
@@ -244,139 +264,217 @@ export default class SmartMemosPlugin extends Plugin {
 
     
     async findFilePath(text: string, regex: RegExp[]) {
-        const fullPath = await this.getAttachmentDir().then((attachmentPath) => {
-            let filename = '';
-            let result: RegExpExecArray | null;
-            for (const reg of regex) {
-                while ((result = reg.exec(text)) !== null) {
-                    filename = normalizePath(decodeURI(result[0])).trim();
-                }
+        console.log('dir text: ', text);
+    
+        let filename = '';
+        let result: RegExpExecArray | null;
+    
+        // Extract the filename using the provided regex patterns
+        for (const reg of regex) {
+            while ((result = reg.exec(text)) !== null) {
+                filename = normalizePath(decodeURI(result[0])).trim();
             }
+        }
     
-            if (filename == '') throw new Error('No file found in the text.');
+        if (filename === '') throw new Error('No file found in the text.');
     
-            const fileInSpecificFolder = filename.includes('/');
-            const AttInRootFolder = attachmentPath === '' || attachmentPath === '/';
-            const AttInCurrentFolder = attachmentPath.startsWith('./');
-            const AttInSpecificFolder = !AttInRootFolder && !AttInCurrentFolder;
+        console.log('file name: ', filename);
     
-            let fullPath = '';
+        // Use the filename directly as the full path
+        const fullPath = filename;
     
-            if (AttInRootFolder || fileInSpecificFolder) fullPath = filename;
-            else {
-                if (AttInSpecificFolder) fullPath = attachmentPath + '/' + filename;
-                if (AttInCurrentFolder) {
-                    const attFolder = attachmentPath.substring(2);
-                    if (attFolder.length == 0) fullPath = this.getCurrentPath() + '/' + filename;
-                    else fullPath = this.getCurrentPath() + '/' + attFolder + '/' + filename;
-                }
-            }
+        console.log('full path: ', fullPath);
     
-            const exists = this.app.vault.getAbstractFileByPath(fullPath) instanceof TAbstractFile;
-            if (exists) return fullPath;
-            else {
-                let path = '';
-                let found = false;
-                this.app.vault.getFiles().forEach((file) => {
-                    if (file.name === filename) {
-                        path = file.path;
-                        found = true;
-                    }
-                });
-                if (found) return path;
-                else throw new Error('File not found');
-            }
-        });
-        return fullPath as string;
+        // Check if the file exists at the constructed path
+        const fileExists = this.app.vault.getAbstractFileByPath(fullPath) instanceof TAbstractFile;
+        if (fileExists) return fullPath;
+    
+        // If not found, search through all files in the vault
+        const allFiles = this.app.vault.getFiles();
+        const foundFile = allFiles.find(file => file.name === filename.split('/').pop());
+        if (foundFile) return foundFile.path;
+    
+        throw new Error('File not found');
     }
-
-	async generateTranscript(audioBuffer: ArrayBuffer, filetype: string) {
+    
+    async generateTranscript(audioBuffer: ArrayBuffer, filetype: string): Promise<string> {
         if (this.settings.apiKey.length <= 1) throw new Error('OpenAI API Key is not provided.');
     
-        // Reference: www.stackoverflow.com/questions/74276173/how-to-send-multipart-form-data-payload-with-typescript-obsidian-library
-        const N = 16 // The length of our random boundry string
-        const randomBoundryString = 'WebKitFormBoundary' + Array(N + 1).join((Math.random().toString(36) + '00000000000000000').slice(2, 18)).slice(0, N)
-        const pre_string = `------${randomBoundryString}\r\nContent-Disposition: form-data; name="file"; filename="audio.mp3"\r\nContent-Type: "application/octet-stream"\r\n\r\n`;
-        const post_string = `\r\n------${randomBoundryString}\r\nContent-Disposition: form-data; name="model"\r\n\r\nwhisper-1\r\n------${randomBoundryString}--\r\n`
-        const pre_string_encoded = new TextEncoder().encode(pre_string);
-        const post_string_encoded = new TextEncoder().encode(post_string);
+        try {
+            // Step 1: Decode Audio Data
+            const decodedAudioData = await this.audioContext.decodeAudioData(audioBuffer);
     
-        // Calculate the size of each chunk
-        const chunkSize = 20 * 1024 * 1024; // 15 MB
+            // Optional: Downsample the audio to 16 kHz for Whisper
+            const targetSampleRate = 16000;
+            const downsampledAudioBuffer = await this.downsampleAudioBuffer(decodedAudioData, targetSampleRate);
     
-        // Calculate the number of chunks
-        const numChunks = Math.ceil(audioBuffer.byteLength / chunkSize);
+            // Step 2: Split Audio Buffer into chunks less than 25 MB
+            const chunkDuration = 600; // in seconds (10 minutes)
+            const audioChunks = this.splitAudioBuffer(downsampledAudioBuffer, chunkDuration);
     
-        if (numChunks < 2) {
-            new Notice(`Transcribing audio...`);
-        } else {
-            new Notice(`Transcribing audio in ${numChunks} chunks. This may take a minute or two...`);
+            let results: string[] = [];
+    
+            for (let i = 0; i < audioChunks.length; i++) {
+                new Notice(`Transcribing chunk #${i + 1} of ${audioChunks.length}...`);
+    
+                // Step 3: Encode Chunk to WAV
+                const wavArrayBuffer = this.encodeAudioBufferToWav(audioChunks[i]);
+    
+                // Check the size of the encoded WAV file
+                const sizeInMB = wavArrayBuffer.byteLength / (1024 * 1024);
+                if (sizeInMB > 24) {
+                    throw new Error('Chunk size exceeds 25 MB limit.');
+                }
+    
+                // Step 4: Send Chunk to Whisper API
+                const formData = new FormData();
+                const blob = new Blob([wavArrayBuffer], { type: 'audio/wav' });
+                formData.append('file', blob, 'audio.wav');
+                formData.append('model', 'whisper-1');
+    
+                const response = await fetch('https://api.openai.com/v1/audio/transcriptions', {
+                    method: 'POST',
+                    headers: {
+                        'Authorization': 'Bearer ' + this.settings.apiKey
+                    },
+                    body: formData
+                });
+    
+                const result = await response.json();
+                if (response.ok && result.text) {
+                    results.push(result.text);
+                } else {
+                    throw new Error(`Error: ${result.error.message}`);
+                }
+    
+                // Wait a bit between requests to avoid rate limits
+                await new Promise(resolve => setTimeout(resolve, 1000));
+            }
+    
+            return results.join(' ');
+    
+        } catch (error) {
+            console.error('Transcription failed:', error);
+            if (error.message.includes('401')) {
+                throw new Error('OpenAI API Key is not valid.');
+            } else if (error.message.includes('400')) {
+                throw new Error('Bad Request. Please check the format of the request.');
+            } else {
+                throw error;
+            }
         }
+    }
 
-
-        // Create an array to store the results
-        let results = [];
-
-        // Process each chunk
-        for (let i = 0; i < numChunks; i++) {
+    async downsampleAudioBuffer(audioBuffer: AudioBuffer, targetSampleRate: number): Promise<AudioBuffer> {
+        const numberOfChannels = audioBuffer.numberOfChannels;
+        const duration = audioBuffer.duration;
     
-            new Notice(`Transcribing chunk #${i + 1}...`);
-
-            // Get the start and end indices for this chunk
-            const start = i * chunkSize;
-            const end = Math.min(start + chunkSize, audioBuffer.byteLength);
+        const offlineContext = new OfflineAudioContext(numberOfChannels, targetSampleRate * duration, targetSampleRate);
     
-            // Extract the chunk from the audio buffer
-            const chunk = audioBuffer.slice(start, end);
+        // Create buffer source
+        const bufferSource = offlineContext.createBufferSource();
+        bufferSource.buffer = audioBuffer;
     
-            // Concatenate the chunk with the pre and post strings
-            const concatenated = await new Blob([pre_string_encoded, chunk, post_string_encoded]).arrayBuffer()
+        // Connect the buffer source to the offline context destination
+        bufferSource.connect(offlineContext.destination);
     
-            const options: RequestUrlParam = {
-                url: 'https://api.openai.com/v1/audio/transcriptions',
-                method: 'POST',
-                contentType: `multipart/form-data; boundary=----${randomBoundryString}`,
-                headers: {
-                    'Authorization': 'Bearer ' + this.settings.apiKey
-                },
-                body: concatenated
-            };
+        // Start rendering
+        bufferSource.start(0);
+        const renderedBuffer = await offlineContext.startRendering();
     
-            const response = await requestUrl(options).catch((error) => { 
-                if (error.message.includes('401')) throw new Error('OpenAI API Key is not valid.');
-                else throw error; 
+        return renderedBuffer;
+    }
+    
+    splitAudioBuffer(audioBuffer: AudioBuffer, chunkDuration: number): AudioBuffer[] {
+        const numberOfChannels = audioBuffer.numberOfChannels;
+        const sampleRate = audioBuffer.sampleRate;
+        const totalSamples = audioBuffer.length;
+    
+        const chunks: AudioBuffer[] = [];
+        let offset = 0;
+        const samplesPerChunk = Math.floor(chunkDuration * sampleRate);
+    
+        while (offset < totalSamples) {
+            const chunkSamples = Math.min(samplesPerChunk, totalSamples - offset);
+            const chunkBuffer = new AudioBuffer({
+                length: chunkSamples,
+                numberOfChannels: numberOfChannels,
+                sampleRate: sampleRate,
             });
     
-            if ('text' in response.json) {
-                // Add the result to the results array
-                results.push(response.json.text);
+            for (let channel = 0; channel < numberOfChannels; channel++) {
+                const channelData = audioBuffer.getChannelData(channel).subarray(offset, offset + chunkSamples);
+                chunkBuffer.copyToChannel(channelData, channel, 0);
             }
-            else throw new Error('Error. ' + JSON.stringify(response.json));
-
-            // Wait for 1 second before processing the next chunk
-            await new Promise(resolve => setTimeout(resolve, 1000));
+    
+            chunks.push(chunkBuffer);
+            offset += chunkSamples;
         }
-        // Return all the results
-        return results.join(' ');
-    }
-
-    async getAttachmentDir() {
-        const activeFile = this.app.workspace.getActiveFile();
-        if (!activeFile) throw new Error('No active file');
-        const dir = this.app.vault.getAvailablePathForAttachments(activeFile.basename, activeFile?.extension, activeFile);  // getAvailablePathForAttachments is undocumented
-        console.log('dir: ', dir);
-        return dir;
+    
+        return chunks;
     }
     
-    getCurrentPath() {
-        const activeFile = this.app.workspace.getActiveFile();
-        if (!activeFile) throw new Error('No active file');
-        const currentPath = activeFile.path.split('/');
-        currentPath.pop();
-        const currentPathString = currentPath.join('/');
-        return currentPathString;
+    encodeAudioBufferToWav(audioBuffer: AudioBuffer): ArrayBuffer {
+        const numChannels = audioBuffer.numberOfChannels;
+        const sampleRate = audioBuffer.sampleRate;
+        const format = 1; // PCM
+        const bitDepth = 16;
+    
+        const numSamples = audioBuffer.length * numChannels;
+        const buffer = new ArrayBuffer(44 + numSamples * 2);
+        const view = new DataView(buffer);
+    
+        /* RIFF identifier */
+        this.writeString(view, 0, 'RIFF');
+        /* file length */
+        view.setUint32(4, 36 + numSamples * 2, true);
+        /* RIFF type */
+        this.writeString(view, 8, 'WAVE');
+        /* format chunk identifier */
+        this.writeString(view, 12, 'fmt ');
+        /* format chunk length */
+        view.setUint32(16, 16, true);
+        /* sample format (raw) */
+        view.setUint16(20, format, true);
+        /* channel count */
+        view.setUint16(22, numChannels, true);
+        /* sample rate */
+        view.setUint32(24, sampleRate, true);
+        /* byte rate (sample rate * block align) */
+        view.setUint32(28, sampleRate * numChannels * bitDepth / 8, true);
+        /* block align (channel count * bytes per sample) */
+        view.setUint16(32, numChannels * bitDepth / 8, true);
+        /* bits per sample */
+        view.setUint16(34, bitDepth, true);
+        /* data chunk identifier */
+        this.writeString(view, 36, 'data');
+        /* data chunk length */
+        view.setUint32(40, numSamples * 2, true);
+    
+        // Write interleaved data
+        let offset = 44;
+        for (let i = 0; i < audioBuffer.length; i++) {
+            for (let channel = 0; channel < numChannels; channel++) {
+                let sample = audioBuffer.getChannelData(channel)[i];
+                // Clip sample
+                sample = Math.max(-1, Math.min(1, sample));
+                // Scale to 16-bit integer
+                sample = sample < 0 ? sample * 0x8000 : sample * 0x7FFF;
+                view.setInt16(offset, sample, true);
+                offset += 2;
+            }
+        }
+    
+        return buffer;
     }
     
+    writeString(view: DataView, offset: number, string: string): void {
+        for (let i = 0; i < string.length; i++) {
+            view.setUint8(offset + i, string.charCodeAt(i));
+        }
+    }
+    
+    
 
 	async generateText(prompt: string, editor: Editor, currentLn: number, contextPrompt?: string) {
         if (prompt.length < 1) throw new Error('Cannot find prompt.');
@@ -500,5 +598,37 @@ class SmartMemosSettingTab extends PluginSettingTab {
                     this.plugin.settings.includeTranscript = value;
                     await this.plugin.saveSettings();
                 }));
+
+        new Setting(containerEl)
+            .setName('Recording File Path')
+            .setDesc('Specify the file path where recordings will be saved. Ex. If you want to put recordings in Resources folder then path is "Resources" (Defaults to root)')
+            .addText(text => text
+                .setPlaceholder('Ex. Resources (if in Resources)')
+                .setValue(this.plugin.settings.recordingFilePath || '')
+                .onChange(async (value) => {
+                    this.plugin.settings.recordingFilePath = value;
+                    await this.plugin.saveSettings();
+                }));
+        
+        new Setting(containerEl)
+        .setName('Save Audio File')
+        .setDesc('Toggle this setting if you want to save/remove the audio file after it has been transcribed.')
+        .addToggle(toggle => toggle
+            .setValue(this.plugin.settings.keepAudio) 
+            .onChange(async (value) => {
+                this.plugin.settings.keepAudio = value;
+                await this.plugin.saveSettings();
+            }));
+    
+        new Setting(containerEl)
+            .setName('Include Audio Player')
+            .setDesc('Toggle this setting if you want the audio file player to be displayed along with the transcription.')
+            .addToggle(toggle => toggle
+                .setValue(this.plugin.settings.includeAudioFileLink) 
+                .onChange(async (value) => {
+                    this.plugin.settings.includeAudioFileLink = value;
+                    await this.plugin.saveSettings();
+                }));
+
 	}
 }

manifest.json

@@ -1,7 +1,7 @@
 {
 	"id": "smart-memos",
 	"name": "Smart Memos",
-	"version": "1.1.3",
+	"version": "1.1.8",
 	"minAppVersion": "0.15.0",
 	"description": "Create personalized and intelligent analysis, summaries, and more for audio recordings that can be imported or spoken directly into a note",
 	"author": "Evan Moscoso",

package.json

@@ -1,6 +1,6 @@
 {
 	"name": "smart-memos",
-	"version": "1.1.3",
+	"version": "1.1.8",
 	"description": "Create personalized and intelligent analysis, summaries, and more for audio recordings that can be imported or spoken directly into a note",
 	"main": "main.js",
 	"scripts": {

styles.css

@@ -191,6 +191,10 @@ body .modal-container .modal.smart-memo-recording {
     animation-play-state: paused;
 }
 
+.smart-memo-keep-audio-container {
+    margin-bottom: -20px;
+}
+
 /* Add this CSS to your existing stylesheet */
 body .modal-container .modal .smart-memo-modal-button:focus,
 body .modal-container .modal .smart-memo-modal-button:focus-visible {

whisper.cpp

@@ -1 +0,0 @@
-Subproject commit c118733a29ad4a984015a5c08fd585086d01087a