A comprehensive guide to AI-powered collaborative drawing and the implementation behind this interactive tool
The AI Co-Drawing application is an interactive collaborative drawing tool that combines human creativity with artificial intelligence. Built with React and powered by Google's Gemini 2.0 Flash model, it allows users to create drawings and have them enhanced, transformed, or evolved through AI assistance.
The application uses Google's latest Gemini 2.0 Flash model with native image generation capabilities. This model can understand both visual input (drawings) and textual instructions to create coherent, contextual enhancements.
Model: gemini-2.0-flash-preview-image-generationInput Modalities: Text + ImageOutput Modalities: Text + ImageMax Resolution: 1024x1024 pixelsThe AI system processes two types of input simultaneously:
1let contents: Content[] = [
2 {
3 role: "USER",
4 parts: [{ inlineData: { data: drawingData, mimeType: "image/png" } }],
5 },
6 {
7 role: "USER",
8 parts: [{ text: `${prompt}. Keep the same minimal line doodle style.` }],
9 },
10];
11
12const response = await ai.models.generateContent({
13 model: "gemini-2.0-flash-preview-image-generation",
14 contents,
15 config: {
16 responseModalities: [Modality.TEXT, Modality.IMAGE],
17 },
18});To maintain visual consistency, the AI is instructed to "keep the same minimal line doodle style" ensuring that enhancements feel like natural extensions of the original drawing rather than complete replacements.
The drawing system uses HTML5 Canvas with a multi-layer approach for optimal performance and functionality.
1const initializeCanvas = () => {
2 const canvas = canvasRef.current;
3 if (!canvas) return;
4 const ctx = canvas.getContext("2d");
5 if (!ctx) return;
6
7 // Set high-resolution canvas size
8 canvas.width = 960;
9 canvas.height = 540;
10
11 // Fill canvas with white background
12 ctx.fillStyle = "#FFFFFF";
13 ctx.fillRect(0, 0, canvas.width, canvas.height);
14};The canvas uses a sophisticated coordinate mapping system to handle different screen sizes and device pixel ratios:
1const getCoordinates = (e: React.MouseEvent | React.TouchEvent) => {
2 const canvas = canvasRef.current;
3 if (!canvas) return { x: 0, y: 0 };
4
5 const rect = canvas.getBoundingClientRect();
6 const scaleX = canvas.width / rect.width;
7 const scaleY = canvas.height / rect.height;
8
9 let clientX, clientY;
10 if ("touches" in e) {
11 clientX = e.touches[0]?.clientX || 0;
12 clientY = e.touches[0]?.clientY || 0;
13 } else {
14 clientX = e.clientX;
15 clientY = e.clientY;
16 }
17
18 return {
19 x: (clientX - rect.left) * scaleX,
20 y: (clientY - rect.top) * scaleY,
21 };
22};The drawing engine supports smooth line rendering with configurable brush properties:
1const draw = (e: React.MouseEvent | React.TouchEvent) => {
2 if (!isDrawing) return;
3
4 const canvas = canvasRef.current;
5 if (!canvas) return;
6 const ctx = canvas.getContext("2d");
7 if (!ctx) return;
8
9 const { x, y } = getCoordinates(e);
10
11 ctx.lineWidth = brushSize;
12 ctx.lineCap = "round";
13 ctx.strokeStyle = penColor;
14 ctx.lineTo(x, y);
15 ctx.stroke();
16};The application uses React hooks for comprehensive state management:
1// Drawing State
2const [isDrawing, setIsDrawing] = useState(false);
3const [penColor, setPenColor] = useState("#000000");
4const [brushSize, setBrushSize] = useState(5);
5
6// AI Generation State
7const [prompt, setPrompt] = useState("");
8const [generatedImage, setGeneratedImage] = useState<string | null>(null);
9const [isLoading, setIsLoading] = useState(false);
10
11// Error Handling
12const [showErrorModal, setShowErrorModal] = useState(false);
13const [errorMessage, setErrorMessage] = useState("");AI-generated images are seamlessly integrated as canvas backgrounds for iterative drawing:
1useEffect(() => {
2 if (generatedImage && canvasRef.current) {
3 const img = new window.Image();
4 img.onload = () => {
5 backgroundImageRef.current = img;
6 drawImageToCanvas();
7 };
8 img.src = generatedImage;
9 }
10}, [generatedImage]);
11
12const drawImageToCanvas = () => {
13 if (!canvasRef.current || !backgroundImageRef.current) return;
14
15 const canvas = canvasRef.current;
16 const ctx = canvas.getContext("2d");
17 if (!ctx) return;
18
19 // Fill with white background first
20 ctx.fillStyle = "#FFFFFF";
21 ctx.fillRect(0, 0, canvas.width, canvas.height);
22
23 // Draw the background image
24 ctx.drawImage(backgroundImageRef.current, 0, 0, canvas.width, canvas.height);
25};Comprehensive touch support for mobile devices with gesture prevention:
1useEffect(() => {
2 const preventTouchDefault = (e: TouchEvent) => {
3 if (isDrawing) {
4 e.preventDefault();
5 }
6 };
7
8 const canvas = canvasRef.current;
9 if (canvas) {
10 canvas.addEventListener("touchstart", preventTouchDefault, { passive: false });
11 canvas.addEventListener("touchmove", preventTouchDefault, { passive: false });
12 }
13
14 return () => {
15 if (canvas) {
16 canvas.removeEventListener("touchstart", preventTouchDefault);
17 canvas.removeEventListener("touchmove", preventTouchDefault);
18 }
19 };
20}, [isDrawing]);The interface follows a consistent design system with dark theme and neutral colors:
The interface adapts to different screen sizes using Tailwind CSS breakpoints:
1{/* Responsive grid layout */}
2<div className="grid grid-cols-1 xl:grid-cols-[1fr_300px] gap-6">
3 {/* Canvas takes full width on mobile, left side on desktop */}
4 <motion.div className="bg-neutral-900 rounded-2xl border border-neutral-800 p-6">
5 <canvas
6 className="w-full h-[400px] md:h-[500px] lg:h-[600px] bg-white cursor-crosshair touch-none"
7 />
8 </motion.div>
9
10 {/* Settings panel stacks below on mobile, right side on desktop */}
11 <motion.div className="bg-neutral-900 rounded-2xl border border-neutral-800 p-6">
12 {/* Settings content */}
13 </motion.div>
14</div>The application requires proper environment setup for API access:
1# Google Generative AI API Key
2# Get your API key from: https://makersuite.google.com/app/apikey
3NEXT_PUBLIC_GOOGLE_API_KEY=your_api_key_hereComprehensive error handling with user-friendly feedback:
1function parseError(error: string) {
2 const regex = /{"error":(.*)}/gm;
3 const m = regex.exec(error);
4 try {
5 const e = m ? m[1] : error;
6 const err = JSON.parse(e);
7 return err.message || error;
8 } catch (e) {
9 return error;
10 }
11}
12
13// In the main component
14} catch (error: any) {
15 console.error("Error submitting drawing:", error);
16 setErrorMessage(error.message || "An unexpected error occurred.");
17 setShowErrorModal(true);
18} finally {
19 setIsLoading(false);
20}API keys are handled securely through environment variables with the NEXT_PUBLIC_ prefix for client-side access. The application includes input validation and sanitization to prevent malicious prompts or canvas manipulations.
This documentation covers the technical implementation of the AI Co-Drawing application. For questions about usage or to contribute improvements, explore the source code or reach out directly.